Datasets

class sylloge.BinaryEADataset(rel_triples, attr_triples, ent_links, dataset_names, folds=None)[source]

Bases: MultiSourceEADataset[DataFrameType, LinkType]

Binary class to get left and right triples easier.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

class sylloge.BinaryParquetEADataset(*, rel_triples: Sequence[DataFrameType], attr_triples: Sequence[DataFrameType], ent_links: LinkType, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, folds: Optional[Sequence[TrainTestValSplit]] = None, backend: Literal['pandas'] = 'pandas')[source]
class sylloge.BinaryParquetEADataset(*, rel_triples: Sequence[DataFrameType], attr_triples: Sequence[DataFrameType], ent_links: LinkType, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, folds: Optional[Sequence[TrainTestValSplit]] = None, backend: Literal['dask'] = 'dask')

Bases: ParquetEADataset[DataFrameType, LinkType], BinaryEADataset[DataFrameType, LinkType]

Binary version of ParquetEADataset.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.CacheableEADataset(*, cache_path: Union[str, Path], use_cache: bool = True, parquet_load_options: Optional[Mapping] = None, parquet_store_options: Optional[Mapping] = None, backend: Literal['pandas'], use_cluster_helper: Literal[False], **init_kwargs)[source]
class sylloge.CacheableEADataset(*, cache_path: Union[str, Path], use_cache: bool = True, parquet_load_options: Optional[Mapping] = None, parquet_store_options: Optional[Mapping] = None, backend: Literal['dask'], use_cluster_helper: Literal[False], **init_kwargs)
class sylloge.CacheableEADataset(*, cache_path: Union[str, Path], use_cache: bool = True, parquet_load_options: Optional[Mapping] = None, parquet_store_options: Optional[Mapping] = None, backend: Literal['pandas'], use_cluster_helper: Literal[True], **init_kwargs)
class sylloge.CacheableEADataset(*, cache_path: Union[str, Path], use_cache: bool = True, parquet_load_options: Optional[Mapping] = None, parquet_store_options: Optional[Mapping] = None, backend: Literal['dask'], use_cluster_helper: Literal[True], **init_kwargs)

Bases: ParquetEADataset[DataFrameType, LinkType]

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)[source]

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

abstract initial_read(backend)[source]

Read data for initialising EADataset.

Return type:

Dict[str, Any]

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.IdMappedEADataset(rel_triples_left, rel_triples_right, attr_triples_left, attr_triples_right, ent_links, entity_mapping, rel_mapping, attr_rel_mapping, attr_mapping, folds=None)[source]

Bases: object

Dataclass holding information of the alignment class with mapping of string to numerical id.

attr_mapping: Dict[str, int]

attribute to id mapping for all attributes

attr_rel_mapping: Dict[str, int]

label to id mapping for all attribute relations

attr_triples_left: ndarray

attribute triples of left knowledge graph

attr_triples_right: ndarray

attribute triples of right knowledge graph

gold standard entity links of alignment

entity_mapping: Dict[str, int]

label to id mapping for all entities

folds: Optional[Sequence[IdMappedTrainTestValSplit]] = None

optional pre-split folds of the gold standard

rel_mapping: Dict[str, int]

label to id mapping for all relations

rel_triples_left: ndarray

relation triples of left knowledge graph

rel_triples_right: ndarray

relation triples of right knowledge graph

class sylloge.MED_BBK(backend: Literal['pandas'] = 'pandas', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None)[source]
class sylloge.MED_BBK(backend: Literal['dask'] = 'dask', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None)

Bases: BinaryZipEADataset[DataFrameType, DataFrameType]

Class containing the MED-BBK dataset.

Published in Zhang, Z. et. al. (2020) An Industry Evaluation of Embedding-based Entity Alignment, COLING

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)[source]

Read data for initialising EADataset.

Return type:

Dict[str, Any]

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.MovieGraphBenchmark(graph_pair: Literal['imdb-tmdb', 'imdb-tvdb', 'tmdb-tvdb', 'multi'] = 'imdb-tmdb', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None, use_cluster_helper: Literal[True] = True)[source]
class sylloge.MovieGraphBenchmark(graph_pair: Literal['imdb-tmdb', 'imdb-tvdb', 'tmdb-tvdb', 'multi'] = 'imdb-tmdb', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None, use_cluster_helper: Literal[False] = False)

Bases: CacheableEADataset[DataFrame, LinkType]

Class containing the movie graph benchmark.

Published in Obraczka, D. et. al. (2021) Embedding-Assisted Entity Resolution for Knowledge Graphs, Proceedings of the 2nd International Workshop on Knowledge Graph Construction co-located with 18th Extended Semantic Web Conference

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)[source]

Read data for initialising EADataset.

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.MultiSourceEADataset(rel_triples, attr_triples, ent_links, dataset_names, folds=None)[source]

Bases: Generic[DataFrameType, LinkType]

Dataset class holding information of the alignment class.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

statistics()[source]

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

class sylloge.OAEI(task='starwars-swg', use_cache=True, cache_path=None)[source]

Bases: BinaryCacheableEADataset[DataFrame, DataFrame]

The OAEI (Ontology Alignment Evaluation Initiative) Knowledge Graph Track tasks contain graphs created from fandom wikis.

Five integration tasks are available:

  • starwars-swg

  • starwars-swtor

  • marvelcinematicuniverse-marvel

  • memoryalpha-memorybeta

  • memoryalpha-stexpanded

More information can be found at the website.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)[source]

Read data for initialising EADataset.

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()[source]

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)[source]

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.OpenEA(graph_pair: Literal['D_W', 'D_Y', 'EN_DE', 'EN_FR'] = 'D_W', size: Literal['15K', '100K'] = '15K', version: Literal['V1', 'V2'] = 'V1', backend: Literal['pandas'] = 'pandas', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None)[source]
class sylloge.OpenEA(graph_pair: Literal['D_W', 'D_Y', 'EN_DE', 'EN_FR'] = 'D_W', size: Literal['15K', '100K'] = '15K', version: Literal['V1', 'V2'] = 'V1', backend: Literal['dask'] = 'dask', use_cache: bool = True, cache_path: Optional[Union[str, Path]] = None)

Bases: BinaryZipEADatasetWithPreSplitFolds[DataFrameType, DataFrameType]

Class containing the OpenEA dataset family.

Published in Sun, Z. et. al. (2020) A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs, Proceedings of the VLDB Endowment

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)

Read data for initialising EADataset.

Return type:

Dict[str, Any]

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.ParquetEADataset(*, rel_triples: Sequence[DataFrameType], attr_triples: Sequence[DataFrameType], ent_links: LinkType, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, folds: Optional[Sequence[TrainTestValSplit]] = None, backend: Literal['pandas'] = 'pandas')[source]
class sylloge.ParquetEADataset(*, rel_triples: Sequence[DataFrameType], attr_triples: Sequence[DataFrameType], ent_links: LinkType, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, folds: Optional[Sequence[TrainTestValSplit]] = None, backend: Literal['dask'] = 'dask')

Bases: MultiSourceEADataset[DataFrameType, LinkType]

Dataset class holding information of the alignment task.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)[source]

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)[source]

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.TrainTestValSplit(train, test, val)[source]

Bases: Generic[LinkType]

Dataclass holding split of gold standard entity links.

test: TypeVar(LinkType, DataFrame, DataFrame, PrefixedClusterHelper)

entity links for testing

train: TypeVar(LinkType, DataFrame, DataFrame, PrefixedClusterHelper)

entity links for training

val: TypeVar(LinkType, DataFrame, DataFrame, PrefixedClusterHelper)

entity links for validation

class sylloge.ZipEADataset(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['pandas'], use_cluster_helper: Literal[False], use_cache: bool = True)[source]
class sylloge.ZipEADataset(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['dask'], use_cluster_helper: Literal[False], use_cache: bool = True)
class sylloge.ZipEADataset(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['pandas'], use_cluster_helper: Literal[True], use_cache: bool = True)
class sylloge.ZipEADataset(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['dask'], use_cluster_helper: Literal[True], use_cache: bool = True)

Bases: CacheableEADataset[DataFrameType, LinkType]

Dataset created from zip file which is downloaded.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)[source]

Read data for initialising EADataset.

Return type:

Dict[str, Any]

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()

class sylloge.ZipEADatasetWithPreSplitFolds(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['pandas'], use_cluster_helper: Literal[False], directory_name_folds: str = '721_5fold', directory_names_individual_folds: Sequence[str] = ('1', '2', '3', '4', '5'), file_name_test_links: str = 'test_links', file_name_train_links: str = 'train_links', file_name_valid_links: str = 'valid_links', use_cache: bool = True)[source]
class sylloge.ZipEADatasetWithPreSplitFolds(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['dask'], use_cluster_helper: Literal[False], directory_name_folds: str = '721_5fold', directory_names_individual_folds: Sequence[str] = ('1', '2', '3', '4', '5'), file_name_test_links: str = 'test_links', file_name_train_links: str = 'train_links', file_name_valid_links: str = 'valid_links', use_cache: bool = True)
class sylloge.ZipEADatasetWithPreSplitFolds(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['pandas'], use_cluster_helper: Literal[True], directory_name_folds: str = '721_5fold', directory_names_individual_folds: Sequence[str] = ('1', '2', '3', '4', '5'), file_name_test_links: str = 'test_links', file_name_train_links: str = 'train_links', file_name_valid_links: str = 'valid_links', use_cache: bool = True)
class sylloge.ZipEADatasetWithPreSplitFolds(*, cache_path: Path, zip_path: str, inner_path: PurePosixPath, dataset_names: Tuple[str, ...], ds_prefix_tuples: Optional[Tuple[str, ...]] = None, file_names_rel_triples: Sequence[str] = ('rel_triples_1', 'rel_triples_2'), file_names_attr_triples: Sequence[str] = ('attr_triples_1', 'attr_triples_2'), file_name_ent_links: str = 'ent_links', backend: Literal['dask'], use_cluster_helper: Literal[True], directory_name_folds: str = '721_5fold', directory_names_individual_folds: Sequence[str] = ('1', '2', '3', '4', '5'), file_name_test_links: str = 'test_links', file_name_train_links: str = 'train_links', file_name_valid_links: str = 'valid_links', use_cache: bool = True)

Bases: ZipEADataset[DataFrameType, LinkType]

Dataset with pre-split folds created from zip file which is downloaded.

property canonical_name: str

A canonical name for this dataset instance.

This includes all the necessary information to distinguish this specific dataset as string. This can be used e.g. to create folders with this dataset name to store results.

Returns:

concise string representation for this dataset instance

create_cache_path(pystow_module, inner_cache_path, cache_path=None)

Use either pystow module or cache_path to create cache path.

Parameters:

  • pystow_module (Module) – module where data is stored

  • inner_cache_path (str) – path relative to pystow/cache path

  • cache_path (Union[str, Path, None]) – alternative to pystow module

Return type:

Path

Returns:

cache path as pathlib.Path

initial_read(backend)[source]

Read data for initialising EADataset.

Return type:

Dict[str, Any]

classmethod read_parquet(path, backend='pandas', use_cluster_helper=True, **kwargs)

Read dataset from parquet files in given path.

This function expects the left/right attribute/relation triples and entity links as well as a dataset_names.txt

Optionally folds are read from a folds directory, with numbered fold subdirectories containing train/test/val links.

Parameters:

  • path (Union[str, Path]) – Directory with files

  • backend (Literal['pandas', 'dask']) – Whether to use pandas or dask for reading

  • kwargs – passed on to the respective read function

  • use_cluster_helper (bool) – if True uses ClusterHelper to load links

Return type:

ParquetEADataset

Returns:

EADataset read from parquet

See also

to_parquet()

statistics()

Provide statistics of datasets.

Return type:

Tuple[List[DatasetStatistics], int]

Returns:

statistics of left dataset, statistics of right dataset and number of gold standard matches

to_parquet(path, **kwargs)

Write dataset to path as several parquet files.

Parameters:

  • path (Union[str, Path]) – directory where dataset will be stored. Will be created if necessary.

  • kwargs – will be handed through to to_parquet functions

See also

read_parquet()