modelrunner.storage.backend package

class HDFStorage(file_or_path, *, mode='read', compression=True)

Bases: StorageBase

Storage that stores data in an HDF file.

Parameters:

• file_or_path (str or Path or Store) – File path to the file/folder or a zarr Store

• mode (str or AccessMode) – The file mode with which the storage is accessed. Determines allowed operations.

• compression (bool) – Whether to store the data in compressed form. Automatically enabled chunked storage.

close()

Closes the storage, potentially writing data to a persistent place.

Return type:

None

extensions: list[str] = ['h5', 'hdf', 'hdf5']

all file extensions supported by this storage

Type:

list of str

is_group(loc)

Determine whether the location is a group.

Parameters:

loc (sequence of str) – A list of strings determining the location in the storage

Returns:

True if the location is a group

Return type:

bool

keys(loc=None)

Return all sub-items defined at a given location.

Parameters:

loc (sequence of str) – A list of strings determining the location in the storage

Returns:

a list of all items defined at this location

Return type:

list

class JSONStorage(path, *, mode='read', simplify=True, **kwargs)

Bases: TextStorageBase

Storage that stores data in a JSON text file.

Note that the data is only written once the storage is closed.

Parameters:

• path (str or Path) – File path to the file

• mode (str or AccessMode) – The file mode with which the storage is accessed. Determines allowed operations.

• simplify (bool) – Flag indicating whether the data is stored in a simplified form

extensions: list[str] = ['json']

all file extensions supported by this storage

Type:

list of str

class MemoryStorage(*, mode='insert')

Bases: StorageBase

Store items in memory.

Parameters:

mode (str or AccessMode) – The file mode with which the storage is accessed. Determines allowed operations.

clear()

Truncate the storage by removing all stored data.

Parameters:

clear_data_shape (bool) – Flag determining whether the data shape is also deleted.

Return type:

None

is_group(loc)

Determine whether the location is a group.

Parameters:

loc (sequence of str) – A list of strings determining the location in the storage

Returns:

True if the location is a group

Return type:

bool

keys(loc)

Return all sub-items defined at a given location.

Parameters:

loc (sequence of str) – A list of strings determining the location in the storage

Returns:

a list of all items defined at this location

Return type:

list

class YAMLStorage(path, *, mode='read', simplify=True, **kwargs)

Bases: TextStorageBase

Storage that stores data in a YAML text file.

Note that the data is only written once the storage is closed.

Parameters:

• path (str or Path) – File path to the file

• mode (str or AccessMode) – The file mode with which the storage is accessed. Determines allowed operations.

• simplify (bool) – Flag indicating whether the data is stored in a simplified form

encode_internal_attrs = True

extensions: list[str] = ['yaml', 'yml']

all file extensions supported by this storage

Type:

list of str

class ZarrStorage(store_or_path, *, mode='read')

Bases: StorageBase

Storage that stores data in an zarr file or database.

Parameters:

• store_or_path (str or Path or Store) – File path to the file/folder or a zarr Store

• mode (str or AccessMode) – The file mode with which the storage is accessed. Determines allowed operations.

property can_update: bool

indicates whether the storage supports updating items.

Type:

Bool

close()

Closes the storage, potentially writing data to a persistent place.

Return type:

None

extensions: list[str] = ['zarr', 'zip', 'sqldb', 'lmdb']

all file extensions supported by this storage

Type:

list of str

is_group(loc, *, ignore_cls=False)

Determine whether the location is a group.

Parameters:

• loc (sequence of str) – A list of strings determining the location in the storage

• ignore_cls (bool)

Returns:

True if the location is a group

Return type:

bool

keys(loc=None)

Return all sub-items defined at a given location.

Parameters:

loc (sequence of str) – A list of strings determining the location in the storage

Returns:

a list of all items defined at this location

Return type:

list

• modelrunner.storage.backend.hdf module
• modelrunner.storage.backend.json module
• modelrunner.storage.backend.memory module
• modelrunner.storage.backend.text_base module
• modelrunner.storage.backend.utils module
• modelrunner.storage.backend.yaml module
• modelrunner.storage.backend.zarr module