dandi.organize

Organize and structure NWB files according to DANDI conventions.

This module provides functionality for organizing neuroscience data files according to DANDI’s file organization schema. Features include: - Automatic path generation from metadata - BIDS-like subject/session organization - Metadata-driven file naming - Validation of organized paths - Support for videos and generic files

Functions

create_dataset_yml_template(filepath)
create_unique_filenames_from_metadata(metadata)	Create unique filenames given metadata
detect_link_type(srcfile, destdir)	Determine what type of links the filesystem will let us make from the file srcfile to the directory destdir.
filter_invalid_metadata_rows(metadata_rows)	Split into two lists - valid and invalid entries
get_obj_id(object_id)	Given full object_id, get its shortened version
is_undefined(value)	Return True if None or an empty container
organize(paths[, dandiset_path, invalid, ...])
organize_external_files(metadata, ...)	Organizes the external_files into the new Dandiset folder structure.
populate_dataset_yml(filepath, metadata)
validate_organized_path(asset_path, ...)

Classes

CopyMode(value)	An enumeration.
FileOperationMode(value)	An enumeration.
OrganizeInvalid(value)	An enumeration.

class dandi.organize.CopyMode(value)

An enumeration.

COPY = 'copy'

HARDLINK = 'hardlink'

MOVE = 'move'

SYMLINK = 'symlink'

copy(old_path: str | Path, new_path: str | Path) → None

class dandi.organize.FileOperationMode(value)

An enumeration.

AUTO = 'auto'

COPY = 'copy'

DRY = 'dry'

HARDLINK = 'hardlink'

MOVE = 'move'

SIMULATE = 'simulate'

SYMLINK = 'symlink'

as_copy_mode() → CopyMode

class dandi.organize.OrganizeInvalid(value)

An enumeration.

FAIL = 'fail'

WARN = 'warn'

dandi.organize.create_dataset_yml_template(filepath)

dandi.organize.create_unique_filenames_from_metadata(metadata: list[dict], required_fields: Sequence[str] | None = None) → list[dict]

Create unique filenames given metadata

Parameters:

• metadata (list of dict) – List of metadata records

• required_fields (sequence of str, optional) – Names from dandi_layout_fields to consider "required_if_not_empty"

Returns:

Adjusted metadata. A copy, which might have removed some metadata fields Do not rely on it being the same

Return type:

list of dict

dandi.organize.detect_link_type(srcfile: str | Path, destdir: str | Path) → FileOperationMode

Determine what type of links the filesystem will let us make from the file srcfile to the directory destdir. If symlinks are allowed, returns "symlink". Otherwise, if hard links are allowed, returns "hardlink". Otherwise, returns "copy".

dandi.organize.filter_invalid_metadata_rows(metadata_rows)

Split into two lists - valid and invalid entries

dandi.organize.get_obj_id(object_id)

Given full object_id, get its shortened version

dandi.organize.is_undefined(value)

Return True if None or an empty container

dandi.organize.organize(paths: Sequence[str], dandiset_path: str | None = None, invalid: OrganizeInvalid = OrganizeInvalid.FAIL, files_mode: FileOperationMode = FileOperationMode.AUTO, devel_debug: bool = False, update_external_file_paths: bool = False, media_files_mode: CopyMode | None = None, required_fields: Sequence[str] | None = None, jobs: int | None = None) → None

dandi.organize.organize_external_files(metadata: list[dict], dandiset_path: str, copy_mode: CopyMode) → None

Organizes the external_files into the new Dandiset folder structure.

Parameters:

• metadata (list) – list of metadata dictionaries created during the call to pynwb_utils._get_pynwb_metadata

• dandiset_path (str) – full path of the main dandiset folder.

• copy_mode (CopyMode)

dandi.organize.populate_dataset_yml(filepath, metadata)

dandi.organize.validate_organized_path(asset_path: str, filepath: Path, dandiset_path: Path) → list[ValidationResult]

Parameters:

• asset_path (str) – The forward-slash-separated path to the asset within its local Dandiset (i.e., relative to the Dandiset’s root)

• filepath (pathlib.Path) – The actual filesystem path of the asset (used to construct ValidationResult objects)

• dandiset_path (pathlib.Path) – The path to the root of the Dandiset (used to construct ValidationResult objects)