dandi.move

Move and rename assets in DANDI Archive.

This module provides functionality for moving and renaming assets both locally and remotely in DANDI Archive instances. Features include: - Local file reorganization - Remote asset path changes - Combined local and remote moves - Conflict resolution (skip, overwrite, error) - Validation of move operations

Module Attributes

AssetPath

A /-separated path to an asset, relative to the root of the Dandiset

Functions

find_dandiset_and_subpath(path)	Find the Dandiset rooted at path or one of its parents, and return the Dandiset along with path made relative to the Dandiset root
move(*srcs, dest[, regex, existing, ...])

Classes

File(path)
Folder(path, relcontents)
LocalMover(dandiset_path, subpath)	A Mover for moving only the assets in a local Dandiset
LocalRemoteMover(local, remote)	A Mover for moving the assets in a local Dandiset and the corresponding remote Dandiset simultaneously.
LocalizedMover()	A Mover for moving only the assets in one location (i.e., either local or remote)
MoveExisting(value)	An enumeration.
MoveWorkOn(value)	An enumeration.
Movement(src, dest[, skip, delete])	A movement/renaming of an asset
Mover()	An abstract base class for calculating and performing asset moves/renames
RemoteMover(dandiset, subpath[, ...])	A Mover for moving only the assets in a remote Dandiset.

Exceptions

AssetMismatchError(mismatches)

exception dandi.move.AssetMismatchError(mismatches: list[str])

class dandi.move.AssetPath

A /-separated path to an asset, relative to the root of the Dandiset

alias of str

class dandi.move.File(path: 'AssetPath')

path: AssetPath

class dandi.move.Folder(path: 'str', relcontents: 'list[str]')

path: str

A /-separated path to an asset folder, relative to the root of the Dandiset

relcontents: list[str]

All file paths under the folder recursively, as /-separated paths relative to the folder path

class dandi.move.LocalMover(dandiset_path: Path, subpath: Path)

A Mover for moving only the assets in a local Dandiset

dandiset_path: Path

The path to the root of the Dandiset

delete(path: AssetPath) → None

Delete the asset at path

get_assets(subpath_only: bool = False) → Iterator[tuple[AssetPath, str]]

Yield all available assets as (asset_path, relpath) pairs, where asset_path is a /-separated path relative to the root of the Dandiset and relpath is a /-separated path to that asset, relative to subpath (For assets outside of subpath, relpath starts with "../"). If subpath_only is true, only assets underneath subpath are returned.

get_path(path: str, is_src: bool = True) → File | Folder

Return the asset or folder of assets at path (relative to subpath) as a File or Folder instance. If there is nothing at the given path, raises NotFoundError.

If the path points to a folder, its relcontents attribute will be populated iff is_src is given.

is_dir(path: AssetPath) → bool

Returns true if the given path points to a directory

is_file(path: AssetPath) → bool

Returns true if the given path points to an asset

move(src: AssetPath, dest: AssetPath) → None

Move the asset at path src to path dest (which can be assumed to not exist)

property placename: str

A description of the mover to show in messages

property status_field: str

Name of the pyout status column

subpath: Path

A relative path denoting the subdirectory of the Dandiset in which we are operating

class dandi.move.LocalRemoteMover(local: LocalMover, remote: RemoteMover)

A Mover for moving the assets in a local Dandiset and the corresponding remote Dandiset simultaneously. It cannot be reused after performing a set of moves.

calculate_moves(*srcs: str, dest: str, existing: MoveExisting) → list[Movement]

Given a sequence of input source paths and a destination path, return a sorted list of all assets that will be moved/renamed

calculate_moves_by_regex(find: str, replace: str, existing: MoveExisting) → list[Movement]

Given a regular expression and a replacement string, return a sorted list of all assets that will be moved/renamed

compare_moves(local_moves: list[Movement], remote_moves: list[Movement]) → None

Given a list of Movement instances calculated by the local and remote Movers, compare them and raise AssetMismatchError if there are any differences.

local: LocalMover

The local Mover

process_movement(m: Movement, dry_run: bool = False) → Iterator[dict[str, str]]

Perform the Movement and yield a dict for each step

remote: RemoteMover

The remote Mover

property status_field: str

Name of the pyout status column.

This specific property should never be used.

property updating_fields: tuple[str, ...]

Names of the pyout fields that are updated as things progress

class dandi.move.LocalizedMover

A Mover for moving only the assets in one location (i.e., either local or remote)

calculate_moves(*srcs: str, dest: str, existing: MoveExisting) → list[Movement]

Given a sequence of input source paths and a destination path, return a sorted list of all assets that will be moved/renamed

calculate_moves_by_regex(find: str, replace: str, existing: MoveExisting) → list[Movement]

Given a regular expression and a replacement string, return a sorted list of all assets that will be moved/renamed

compile_moves(moves: dict[AssetPath, AssetPath], existing: MoveExisting) → list[Movement]

Given a dict mapping source paths to destination paths, produce a sorted list of Movement instances.

abstract delete(path: AssetPath) → None

Delete the asset at path

abstract get_assets(subpath_only: bool = False) → Iterator[tuple[AssetPath, str]]

Yield all available assets as (asset_path, relpath) pairs, where asset_path is a /-separated path relative to the root of the Dandiset and relpath is a /-separated path to that asset, relative to subpath (For assets outside of subpath, relpath starts with "../"). If subpath_only is true, only assets underneath subpath are returned.

abstract get_path(path: str, is_src: bool = True) → File | Folder

Return the asset or folder of assets at path (relative to subpath) as a File or Folder instance. If there is nothing at the given path, raises NotFoundError.

If the path points to a folder, its relcontents attribute will be populated iff is_src is given.

abstract is_dir(path: AssetPath) → bool

Returns true if the given path points to a directory

abstract is_file(path: AssetPath) → bool

Returns true if the given path points to an asset

abstract move(src: AssetPath, dest: AssetPath) → None

Move the asset at path src to path dest (which can be assumed to not exist)

abstract property placename: str

A description of the mover to show in messages (either "local" or "remote")

process_movement(m: Movement, dry_run: bool = False) → Iterator[dict[str, str]]

Perform the Movement and yield a dict for each step

resolve(path: str) → tuple[AssetPath, bool]

Convert an input path (relative to subpath, possibly starting with ../) to a /-separated path relative to the root of the Dandiset, plus a boolean that is true iff path ended with a slash

subpath: Path

A relative path denoting the subdirectory of the Dandiset in which we are operating

class dandi.move.MoveExisting(value)

An enumeration.

ERROR = 'error'

OVERWRITE = 'overwrite'

SKIP = 'skip'

class dandi.move.MoveWorkOn(value)

An enumeration.

AUTO = 'auto'

BOTH = 'both'

LOCAL = 'local'

REMOTE = 'remote'

class dandi.move.Movement(src: AssetPath, dest: AssetPath, skip: bool = False, delete: bool = False)

A movement/renaming of an asset

delete: bool = False

Whether to delete the asset at the destination before moving

dest: AssetPath

The asset’s destination path

property dest_exists: bool

True iff an asset already exists at the destination

skip: bool = False

Whether to skip this operation because an asset already exists at the destination

src: AssetPath

The asset’s original path

class dandi.move.Mover

An abstract base class for calculating and performing asset moves/renames

abstract calculate_moves(*srcs: str, dest: str, existing: MoveExisting) → list[Movement]

Given a sequence of input source paths and a destination path, return a sorted list of all assets that will be moved/renamed

abstract calculate_moves_by_regex(find: str, replace: str, existing: MoveExisting) → list[Movement]

Given a regular expression and a replacement string, return a sorted list of all assets that will be moved/renamed

property columns: tuple[str, ...]

Names of the columns in the pyout display

abstract process_movement(m: Movement, dry_run: bool = False) → Iterator[dict[str, str]]

Perform the Movement and yield a dict for each step

process_moves_debug(plan: list[Movement], dry_run: bool = False) → Iterator[Iterator[dict]]

For each Movement in plan, yield an iterator of dicts to print for each step of the movement operation.

process_moves_pyout(plan: list[Movement], dry_run: bool = False) → Iterator[dict]

Yield a dict to pass to pyout for each Movement in plan

abstract property status_field: str

Name of the pyout status column (either "local" or "remote")

property updating_fields: tuple[str, ...]

Names of the pyout fields that are updated as things progress

class dandi.move.RemoteMover(dandiset: RemoteDandiset, subpath: Path, local_dandiset_path: Path | None = None)

A Mover for moving only the assets in a remote Dandiset. It cannot be reused after performing a set of moves.

assets: dict[AssetPath, RemoteAsset]

A collection of all assets in the Dandiset, keyed by their paths

dandiset: RemoteDandiset

The client object for the remote Dandiset being operated on

delete(path: AssetPath) → None

Delete the asset at path

get_assets(subpath_only: bool = False) → Iterator[tuple[AssetPath, str]]

Yield all available assets as (asset_path, relpath) pairs, where asset_path is a /-separated path relative to the root of the Dandiset and relpath is a /-separated path to that asset, relative to subpath (For assets outside of subpath, relpath starts with "../"). If subpath_only is true, only assets underneath subpath are returned.

get_path(path: str, is_src: bool = True) → File | Folder

Return the asset or folder of assets at path (relative to subpath) as a File or Folder instance. If there is nothing at the given path, raises NotFoundError.

If the path points to a folder, its relcontents attribute will be populated iff is_src is given.

is_dir(path: AssetPath) → bool

Returns true if the given path points to a directory

is_file(path: AssetPath) → bool

Returns true if the given path points to an asset

local_dandiset_path: Path | None = None

The dandiset_path of the corresponding LocalMover when inside a LocalRemoteMover

move(src: AssetPath, dest: AssetPath) → None

Move the asset at path src to path dest (which can be assumed to not exist)

property placename: str

A description of the mover to show in messages

property status_field: str

Name of the pyout status column

subpath: Path

A relative path denoting the subdirectory of the Dandiset in which we are operating

dandi.move.find_dandiset_and_subpath(path: Path) → tuple[Dandiset, Path]

Find the Dandiset rooted at path or one of its parents, and return the Dandiset along with path made relative to the Dandiset root

dandi.move.move(*srcs: str, dest: str, regex: bool = False, existing: MoveExisting = MoveExisting.ERROR, dandi_instance: str | DandiInstance = 'dandi', dandiset: str | Path | None = None, work_on: MoveWorkOn = MoveWorkOn.AUTO, devel_debug: bool = False, jobs: int | None = None, dry_run: bool = False) → None