dandi.dandiarchive#

This module provides functionality for parsing URLs and other resource identifiers for Dandisets & assets on Dandi Archive servers and for fetching the objects to which the URLs refer. See Resource Identifiers for a list of accepted URL formats.

Basic operation begins by calling parse_dandi_url() on a URL in order to acquire a ParsedDandiURL instance, which can then be used to obtain the Dandiset and/or assets specified in the URL. Call an instance’s get_dandiset() and/or get_assets() to get the assets, passing in a DandiAPIClient for the appropriate Dandi Archive API instance; an unauthenticated client pointing to the correct instance can be acquired via the get_client() method. As a convenience, one can acquire a client, the Dandiset, and an iterator of all assets by using the navigate() context manager like so:

parsed_url = parse_dandi_url("https://...")
with parsed_url.navigate() as (client, dandiset, assets):
    ...
# The client's session is closed when the context manager exits.

As a further convenience, a URL can be parsed and navigated in one fell swoop using the navigate_url() function.

class dandi.dandiarchive.ParsedDandiURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None)[source]#

Bases: ABC

Parsed representation of a URL pointing to a Dandi Archive resource (Dandiset or asset(s)). Subclasses must implement get_assets().

Most methods take a client: DandiAPIClient argument, which must be a DandiAPIClient object for querying instance (This is not checked). Such a client instance can be obtained by calling get_client(), or an appropriate pre-existing client instance can be passed instead.

instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

property api_url: Url[Url]#

The base URL of the Dandi API service, without a trailing slash

get_client() → DandiAPIClient[source]#

Returns an unauthenticated DandiAPIClient for instance

get_dandiset(client: DandiAPIClient, lazy: bool = True) → RemoteDandiset | None[source]#

Returns information about the specified (or default) version of the specified Dandiset. Returns None if the URL did not contain a Dandiset identifier.

If lazy is true, a “lazy” RemoteDandiset instance is returned, with no requests made until any data is actually required.

abstract get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Returns an iterator of asset structures for the assets referred to by or associated with the URL. For a URL pointing to just a Dandiset, this is the set of all assets in the given or default version of the Dandiset. For a URL that specifies a specific asset or collection of assets in a Dandiset, this is all of those assets.

When multiple assets are returned, they can be sorted by a given field by passing the name of that field as the order parameter. The accepted field names are "created", "modified", and "path". Prepend a hyphen to the field name to reverse the sort order.

If strict is true, then fetching assets for a URL that refers to a nonexistent resource will raise a NotFoundError; if it is false, the method will instead return an empty iterator.

get_asset_ids(client: DandiAPIClient) → Iterator[str][source]#

Returns an iterator of IDs of the assets referred to by or associated with the URL

navigate(*, strict: bool = False, authenticate: bool | None = None) → Iterator[tuple[DandiAPIClient, RemoteDandiset | None, Iterable[BaseRemoteAsset]]][source]#

A context manager that returns a triple of a DandiAPIClient (with an open session that is closed when the context manager closes), the return value of get_dandiset(), and the return value of get_assets().

If strict is true, then get_dandiset() is called with lazy=False and get_assets() is called with strict=True; if strict is false, the opposite occurs.

If authenticate is true, then dandi_authenticate() will be called on the client before returning it. If it is None (the default), the method will only be called if the URL requires authentication (e.g., if the resource(s) are embargoed).

Changed in version 0.35.0: authenticate added

class dandi.dandiarchive.DandisetURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None)[source]#

Bases: ParsedDandiURL

Parsed from a URL that only refers to a Dandiset (possibly with a version)

get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Returns all assets in the Dandiset

instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

class dandi.dandiarchive.SingleAssetURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None)[source]#

Bases: ParsedDandiURL

Superclass for parsed URLs that refer to a single asset

instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

class dandi.dandiarchive.MultiAssetURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, path: str)[source]#

Bases: ParsedDandiURL

Superclass for parsed URLs that refer to multiple assets

path: str#
class dandi.dandiarchive.BaseAssetIDURL(instance: DandiInstance, asset_id: str)[source]#

Bases: SingleAssetURL

Parsed from a URL that refers to an asset by ID and does not include the Dandiset ID

dandiset_id: None = None#

The ID of the Dandiset given in the URL

version_id: None = None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

asset_id: str#
get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Yields the asset with the given ID. If the asset does not exist, then a NotFoundError is raised if strict is true, and nothing is yielded if strict is false.

get_asset_ids(client: DandiAPIClient) → Iterator[str][source]#

Yields the ID of the asset (regardless of whether it exists)

navigate(*, strict: bool = False, authenticate: bool | None = None) → Iterator[tuple[DandiAPIClient, RemoteDandiset | None, Iterable[BaseRemoteAsset]]][source]#

A context manager that returns a triple of a DandiAPIClient (with an open session that is closed when the context manager closes), the return value of get_dandiset(), and the return value of get_assets().

If strict is true, then get_dandiset() is called with lazy=False and get_assets() is called with strict=True; if strict is false, the opposite occurs.

If authenticate is true, then dandi_authenticate() will be called on the client before returning it. If it is None (the default), the method will only be called if the URL requires authentication (e.g., if the resource(s) are embargoed).

Changed in version 0.35.0: authenticate added

class dandi.dandiarchive.AssetIDURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, asset_id: str)[source]#

Bases: SingleAssetURL

Parsed from a URL that refers to an asset by ID and includes the Dandiset ID

asset_id: str#
get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Yields the asset with the given ID. If the Dandiset or asset does not exist, then a NotFoundError is raised if strict is true, and nothing is yielded if strict is false.

get_asset_ids(client: DandiAPIClient) → Iterator[str][source]#

Yields the ID of the asset (regardless of whether it exists)

class dandi.dandiarchive.AssetPathPrefixURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, path: str)[source]#

Bases: MultiAssetURL

Parsed from a URL that refers to a collection of assets by path prefix

get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Returns the assets whose paths start with path. If strict is true and there are no such assets, raises NotFoundError.

Changed in version 0.54.0: NotFoundError will now be raised if strict is true and there are no such assets.

path: str#
instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

class dandi.dandiarchive.AssetItemURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, path: str)[source]#

Bases: SingleAssetURL

Parsed from a URL that refers to a specific asset by path

path: str#
get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Yields the asset whose path equals path. If there is no such asset:

  • If strict is true, a NotFoundError is raised.

  • If strict is false and the path happens to be the path to an asset directory, a ValueError is raised indicating that the user left off a trailing slash.

  • Otherwise, nothing is yielded.

class dandi.dandiarchive.AssetFolderURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, path: str)[source]#

Bases: MultiAssetURL

Parsed from a URL that refers to a collection of assets by folder path

get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Returns all assets under the folder at path. If the folder does not exist and strict is true, raises NotFoundError; otherwise, if the folder does not exist and strict is false, yields nothing.

Changed in version 0.54.0: NotFoundError will now be raised if strict is true and there is no such folder.

path: str#
instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

class dandi.dandiarchive.AssetGlobURL(instance: DandiInstance, dandiset_id: str | None, version_id: str | None, path: str)[source]#

Bases: MultiAssetURL

New in version 0.54.0.

Parsed from a URL that refers to a collection of assets by a path glob

get_assets(client: DandiAPIClient, order: str | None = None, strict: bool = False) → Iterator[BaseRemoteAsset][source]#

Returns all assets whose paths match the glob pattern path. If strict is true and there are no such assets, raises NotFoundError.

Changed in version 0.54.0: NotFoundError will now be raised if strict is true and there are no such assets.

path: str#
instance: DandiInstance#

The Dandi Archive instance that the URL points to

dandiset_id: str | None#

The ID of the Dandiset given in the URL

version_id: str | None#

The version of the Dandiset, if specified. If this is not set, the version will be defaulted using the rules described under DandiAPIClient.get_dandiset().

dandi.dandiarchive.navigate_url(url: str, *, strict: bool = False, authenticate: bool | None = None) → Iterator[tuple[DandiAPIClient, RemoteDandiset | None, Iterable[BaseRemoteAsset]]][source]#

A context manager that takes a URL pointing to a DANDI Archive and returns a triple of a DandiAPIClient (with an open session that is closed when the context manager closes), the Dandiset identified in the URL (if any), and the assets specified by the URL (or, if no specific assets were specified, all assets in the Dandiset).

Changed in version 0.35.0: authenticate added

Parameters:

  • url (str) – URL which might point to a Dandiset, folder, or asset(s)

  • strict (bool) – If true, then get_dandiset() is called with lazy=False and get_assets() is called with strict=True; if false, the opposite occurs.

  • authenticate – If true, then dandi_authenticate() will be called on the client before returning it. If None (the default), the method will only be called if the URL requires authentication (e.g., if the resource(s) are embargoed).

Returns:

Context manager that yields a (client, dandiset, assets) tuple; client will have a session established for the duration of the context

dandi.dandiarchive.parse_dandi_url(url: str, *, map_instance: bool = True, glob: bool = False) → ParsedDandiURL#

Parse a Dandi Archive URL and return a ParsedDandiURL instance. See Resource Identifiers for the supported URL formats.

New in version 0.54.0: glob parameter added

Parameters:

glob (bool) – if true, certain URL formats will be parsed into AssetGlobURL instances

Raises:

UnknownURLError – if the URL is not one of the above

dandi.dandiarchive.follow_redirect(url: str) → str#

Resolve the given URL by following all redirects.

Raises:

  • NotFoundError – if a 404 response is returned

  • FailedToConnectError – if a response other than 200, 400, 404, or one of the statuses in RETRY_STATUSES is returned