dandi.organize

ATM primarily a sandbox for some functionality for dandi organize

Functions

create_dataset_yml_template(filepath)
create_unique_filenames_from_metadata(metadata)	Create unique filenames given metadata
detect_link_type(srcfile, destdir)	Determine what type of links the filesystem will let us make from the file srcfile to the directory destdir.
filter_invalid_metadata_rows(metadata_rows)	Split into two lists - valid and invalid entries
get_obj_id(object_id)	Given full object_id, get its shortened version
is_undefined(value)	Return True if None or an empty container
organize(paths[, dandiset_path, invalid, ...])
organize_external_files(metadata, ...)	Organizes the external_files into the new Dandiset folder structure.
populate_dataset_yml(filepath, metadata)
validate_organized_path(asset_path, ...)	param str asset_path:

Classes

CopyMode(value)	An enumeration.
FileOperationMode(value)	An enumeration.
OrganizeInvalid(value)	An enumeration.

class dandi.organize.CopyMode(value)

An enumeration.

COPY = 'copy'

HARDLINK = 'hardlink'

MOVE = 'move'

SYMLINK = 'symlink'

copy(old_path: str | Path, new_path: str | Path) → None

class dandi.organize.FileOperationMode(value)

An enumeration.

AUTO = 'auto'

COPY = 'copy'

DRY = 'dry'

HARDLINK = 'hardlink'

MOVE = 'move'

SIMULATE = 'simulate'

SYMLINK = 'symlink'

as_copy_mode() → CopyMode

class dandi.organize.OrganizeInvalid(value)

An enumeration.

FAIL = 'fail'

WARN = 'warn'

dandi.organize.create_dataset_yml_template(filepath)

dandi.organize.create_unique_filenames_from_metadata(metadata: list[dict], required_fields: Sequence[str] | None = None) → list[dict]

Create unique filenames given metadata

Parameters:

• metadata (list of dict) – List of metadata records

• required_fields (sequence of str, optional) – Names from dandi_layout_fields to consider "required_if_not_empty"

Returns:

Adjusted metadata. A copy, which might have removed some metadata fields Do not rely on it being the same

Return type:

list of dict

dandi.organize.detect_link_type(srcfile: str | Path, destdir: str | Path) → FileOperationMode

Determine what type of links the filesystem will let us make from the file srcfile to the directory destdir. If symlinks are allowed, returns "symlink". Otherwise, if hard links are allowed, returns "hardlink". Otherwise, returns "copy".

dandi.organize.filter_invalid_metadata_rows(metadata_rows)

Split into two lists - valid and invalid entries

dandi.organize.get_obj_id(object_id)

Given full object_id, get its shortened version

dandi.organize.is_undefined(value)

Return True if None or an empty container

dandi.organize.organize(paths: Sequence[str], dandiset_path: str | None = None, invalid: OrganizeInvalid = OrganizeInvalid.FAIL, files_mode: FileOperationMode = FileOperationMode.AUTO, devel_debug: bool = False, update_external_file_paths: bool = False, media_files_mode: CopyMode | None = None, required_fields: Sequence[str] | None = None, jobs: int | None = None) → None

dandi.organize.organize_external_files(metadata: list[dict], dandiset_path: str, copy_mode: CopyMode) → None

Organizes the external_files into the new Dandiset folder structure.

Parameters:

• metadata (list) – list of metadata dictionaries created during the call to pynwb_utils._get_pynwb_metadata

• dandiset_path (str) – full path of the main dandiset folder.

• copy_mode (CopyMode) –

dandi.organize.populate_dataset_yml(filepath, metadata)

dandi.organize.validate_organized_path(asset_path: str, filepath: Path, dandiset_path: Path) → list[ValidationResult]

Parameters:

• asset_path (str) – The forward-slash-separated path to the asset within its local Dandiset (i.e., relative to the Dandiset’s root)

• filepath (pathlib.Path) – The actual filesystem path of the asset (used to construct ValidationResult objects)

• dandiset_path (pathlib.Path) – The path to the root of the Dandiset (used to construct ValidationResult objects)