Linkages#

Linkages are tools for building relationships that span multiple tables.

You can use linkages to act as indexes to relate the rows from two different tables which share common values. In fact, linkages are a bit more general than that: they can relate rows based on computed values as well.

class quivr.Linkage(left_table, right_table, left_keys, right_keys)#

A Linkage is a mapping of rows across two Tables.

The mapping is defined by a pair of arrays, one for each table, that contain common values.

The Linkage can be used to iterate over all the unique values in the linkage columns, and to select the rows from each table that match a particular value.

Parameters:

  • left_table (~LeftTable) – The left table in the linkage.

  • right_table (~RightTable) – The right table in the linkage.

  • left_keys (Array) – The array of keys from the left table.

  • right_keys (Array) – The array of keys from the right table.

Variables:

  • left_table (Table) – The left table in the linkage.

  • right_table (Table) – The right table in the linkage.

  • left_keys (pyarrow.Array) – The array of keys for the left table.

  • right_keys (pyarrow.Array) – The array of keys for the right table.

select_left(val)#

Select the rows from the left table that match the given value.

If the value is not present in the left table, then an empty table is returned.

Return type:

~LeftTable

select_right(val)#

Select the rows from the right table that match the given value.

If the value is not present in the right table, then an empty table is returned.

Return type:

~RightTable

select(val)#

Select the rows from both tables that match the given value.

If the value is not present in either table, then an empty table is returned for that table.

Return type:

tuple[~LeftTable, ~RightTable]

iterate()#

Returns an iterator over all the unique values in the linkage, and the rows from each table that match that value.

Return type:

Iterator[tuple[Scalar, ~LeftTable, ~RightTable]]

class quivr.MultiKeyLinkage(left_table, right_table, left_keys, right_keys)#

A MultiKeyLinkage links two tables using multiple arrays for composite key relationships.

The linkage is defined by a pair of dictionaries, one for each table, which define the composite keys.

The dictionaries must have the same keys, and the arrays must:

  • be identically typed under the same keys

  • have no null values

  • be the same length as the associated table

Example:
>>> from quivr import *
>>> class Positions(Table):
...     x = Float32Column()
...     y = Float32Column()
...     time = TimestampColumn(unit="s")
...     id = UInt32Column()
...
>>> class Velocities(tables.Table):
...     vx = Float32Column()
...     vy = Float32Column()
...     time = TimestampColumn(unit="s")
...     id = UInt32Column()
...
>>> positions = Positions.from_data(
...     x=[0.0, 1.0, 2.0, 3.0, 4.0],
...     y=[0.0, 1.0, 2.0, 3.0, 4.0],
...     time=[0, 1, 2, 3, 4],
...     id=[0, 1, 1, 2, 2],
... )
>>> velocities = Velocities.from_data(
...     vx=[0.0, 1.0, 2.0, 3.0, 4.0],
...     vy=[0.0, 1.0, 2.0, 3.0, 4.0],
...     time=[0, 1, 2, 3, 4],
...     id=[0, 1, 1, 2, 2],
... )
>>> linkage = MultiKeyLinkage(
...     positions,
...     velocities,
...     {"id": positions.id, "time": positions.time},
...     {"id": velocities.id, "time": velocities.time},
... )
>>> for val, left, right in sorted(linkage, key=lambda x: (x[0][1].as_py())):
...     print(val, left, right)
[('id', 0), ('time', datetime.datetime(1970, 1, 1, 0, 0))] Positions(size=1) Velocities(size=1)
[('id', 1), ('time', datetime.datetime(1970, 1, 1, 0, 0, 1))] Positions(size=1) Velocities(size=1)
[('id', 1), ('time', datetime.datetime(1970, 1, 1, 0, 0, 2))] Positions(size=1) Velocities(size=1)
[('id', 2), ('time', datetime.datetime(1970, 1, 1, 0, 0, 3))] Positions(size=1) Velocities(size=1)
[('id', 2), ('time', datetime.datetime(1970, 1, 1, 0, 0, 4))] Positions(size=1) Velocities(size=1)
Parameters:

  • left_table (~LeftTable) – The left table to link.

  • right_table (~RightTable) – The right table to link.

  • left_keys (dict[str, Array]) – A dictionary of key names to arrays of values. The arrays must be the same length as the left table, and must not contain null values. The key names must be the same as the right keys.

  • right_keys (dict[str, Array]) – A dictionary of key names to arrays of values. The arrays must be the same length as the right table, and must not contain null values. The key names must be the same as the left keys.

Raises:

ValueError – If the keys do not match the requirements above.

key(**kwargs)#

Returns a composite key scalar for the given values.

Example:
>>> from quivr import *
>>> class Positions(Table):
...     x = Float32Column()
...     y = Float32Column()
...     time = TimestampColumn(unit="s")
...     id = UInt32Column()
...
>>> class Velocities(tables.Table):
...     vx = Float32Column()
...     vy = Float32Column()
...     time = TimestampColumn(unit="s")
...     id = UInt32Column()
...
>>> positions = Positions.from_data(
...     x=[0.0, 1.0, 2.0, 3.0, 4.0],
...     y=[0.0, 1.0, 2.0, 3.0, 4.0],
...     time=[0, 1, 2, 3, 4],
...     id=[0, 1, 1, 2, 2],
... )
>>> velocities = Velocities.from_data(
...     vx=[0.0, 1.0, 2.0, 3.0, 4.0],
...     vy=[0.0, 1.0, 2.0, 3.0, 4.0],
...     time=[0, 1, 2, 3, 4],
...     id=[0, 1, 1, 2, 2],
... )
>>> linkage = MultiKeyLinkage(
...     positions,
...     velocities,
...     {"time": positions.time, "id": positions.id},
...     {"time": velocities.time, "id": velocities.id},
... )
>>> key = linkage.key(time=1, id=1)
>>> key
<pyarrow.StructScalar: [('time', datetime.datetime(1970, 1, 1, 0, 0, 1)), ('id', 1)]>
>>> linkage[key]
(Positions(size=1), Velocities(size=1))
Parameters:

kwargs – The values for the composite key.

Raises:

ValueError – If the keys do not match the linkage keys.

Return type:

Scalar

quivr.combine_linkages(links)#

Combine a list of linkages into a single linkage.

The combined linkage will concatenate the left and right tables of the source linkages, and then build an index on the combined tables, using the same keys as the source linkages.

All the input linkages must have the same key names and table types. All the tables that comprise the input linkages must have identical attribute values, including for any nested subtables.

Parameters:

links (Iterable[Linkage[~LeftTable, ~RightTable]]) – The linkages to concatenate.

Raises:

  • LinkageCombinationError – If the linkages do not have the same key names or if the tables are not concatenatable.

  • ValueError – If less than two linkages are provided.

Return type:

Linkage[~LeftTable, ~RightTable]

Returns:

A linkage that combines the input linkages.

quivr.combine_multilinkages(links)#

Combine a list of MultiKeyLinkages into a single MultiKeyLinkage.

The combined linkage will concatenate the left and right tables of the source linkages, and then build an index on the combined tables, using the same keys as the source linkages.

All the input linkages must have the same key structure (both names and types), and must use the same table classes. All the tables that comprise the input linkages must have identical attribute values, including for any nested subtables.

Parameters:

links (Iterable[MultiKeyLinkage[~LeftTable, ~RightTable]]) – The linkages to concatenate.

Raises:

  • LinkageCombinationError – If the linkages do not have the same key names or if the tables are not concatenatable.

  • ValueError – If less than two linkages are provided.

Return type:

MultiKeyLinkage[~LeftTable, ~RightTable]

Returns:

A linkage that combines the input linkages.

quivr.linkage.LeftTable = TypeVar(LeftTable, bound=Table)#

Type:    TypeVar

Invariant TypeVar bound to quivr.tables.Table.

quivr.linkage.RightTable = TypeVar(RightTable, bound=Table)#

Type:    TypeVar

Invariant TypeVar bound to quivr.tables.Table.