dandi.dandiapi
#
This module provides functionality for interacting with a Dandi Archive server
via the REST API. Interaction begins with the creation of a DandiAPIClient
instance, which can be used to retrieve RemoteDandiset
objects (representing
Dandisets on the server) and BaseRemoteAsset
objects (representing assets
without any data associating them with their Dandisets). RemoteDandiset
objects can, in turn, be used to retrieve RemoteAsset
objects (representing
assets associated with Dandisets). Aside from DandiAPIClient
, none of these
classes should be instantiated directly by the user.
All operations that merely fetch data from the server can be done without
authenticating, but any operation that writes, uploads, modifies, or deletes
data requires the user to authenticate the DandiAPIClient
instance by
supplying an API key either when creating the instance or by calling the
authenticate()
or dandi_authenticate()
method.
Example code for printing the metadata of all assets with “two-photon” in their
metadata.measurementTechnique[].name
for the latest published version of
every Dandiset:
import json
from dandi.dandiapi import DandiAPIClient
with DandiAPIClient.for_dandi_instance("dandi") as client:
for dandiset in client.get_dandisets():
if dandiset.most_recent_published_version is None:
continue
latest_dandiset = dandiset.for_version(dandiset.most_recent_published_version)
for asset in latest_dandiset.get_assets():
metadata = asset.get_metadata()
if any(
mtt is not None and "two-photon" in mtt.name
for mtt in (metadata.measurementTechnique or [])
):
print(json.dumps(metadata.json_dict(), indent=4))
# Can be used to also download the asset:
# asset.download(pathlib.Path(dandiset.identifier, asset.path))
Example code for accessing asset files as regular Python file objects without downloading their entire content locally. Such file objects could then be passed to functions of pynwb etc.
from dandi.dandiapi import DandiAPIClient
dandiset_id = "000006" # ephys dataset from the Svoboda Lab
filepath = "sub-anm372795/sub-anm372795_ses-20170718.nwb" # 450 kB file
with DandiAPIClient() as client:
asset = client.get_dandiset(dandiset_id, "draft").get_asset_by_path(filepath)
# https://dandi.readthedocs.io/en/latest/modref/dandiapi.html#dandi.dandiapi.BaseRemoteBlobAsset.as_readable
# provides file-like object which uses fsspec to provide sparse access to content
# of the file on S3:
with asset.as_readable().open() as f:
print(f.read(4))
f.seek(100)
print(f.read(4))
You can see more usages of DANDI API to assist with data streaming at PyNWB: Streaming NWB files.
Client#
- class dandi.dandiapi.RESTFullAPIClient(api_url: str, session: requests.Session | None = None, headers: dict | None = None)[source]#
Base class for a JSON-based HTTP(S) client for interacting with a given base API URL.
All request methods can take either an absolute URL or a slash-separated path; in the latter case, the path is appended to the base API URL (separated by a slash) in order to determine the actual URL to make the request of.
RESTFullAPIClient
instances are usable as context managers, in which case they will close their associated session on exit.- delete(path: str, **kwargs: Any) Any [source]#
Convenience method to call
request()
with the ‘DELETE’ HTTP method.
- get(path: str, **kwargs: Any) Any [source]#
Convenience method to call
request()
with the ‘GET’ HTTP method.
- get_url(path: str) str [source]#
Append a slash-separated
path
to the instance’s base URL. The two components are separated by a single slash, removing any excess slashes that would be present after naïve concatenation.If
path
is already an absolute URL, it is returned unchanged.
- page_size: int | None#
Default number of items to request per page when paginating (
None
means to use the server’s default)
- page_workers: int#
How many pages to fetch at once when parallelizing pagination
- paginate(path: str, page_size: int | None = None, params: dict | None = None) Iterator [source]#
Paginate through the resources at the given path: GET the path, yield the values in the
"results"
key, and repeat with the URL in the"next"
key until it isnull
.If the first
"next"
key is the same as the initially-requested URL but with thepage
query parameter set to2
, then the remaining pages are fetched concurrently in separate threads,page_workers
(default 5) at a time. This behavior requires the initial response to contain a"count"
key giving the number of items across all pages.- Parameters:
page_size – If non-
None
, overrides the client’spage_size
attribute for this sequence of pages
- patch(path: str, **kwargs: Any) Any [source]#
Convenience method to call
request()
with the ‘PATCH’ HTTP method.
- post(path: str, **kwargs: Any) Any [source]#
Convenience method to call
request()
with the ‘POST’ HTTP method.
- put(path: str, **kwargs: Any) Any [source]#
Convenience method to call
request()
with the ‘PUT’ HTTP method.
- request(method: str, path: str, params: dict | None = None, data: Any = None, files: dict | None = None, json: Any = None, headers: dict | None = None, json_resp: bool = True, retry_statuses: Sequence[int] = (), retry_if: Callable[[requests.Response], Any] | None = None, **kwargs: Any) Any [source]#
This method looks up the appropriate method, constructs a request URL from the base URL, path, and parameters, and then sends the request. If the method is unknown or if the path is not found, an exception is raised; otherwise, a JSON object is returned with the response.
This is a convenience method to use when making basic requests that do not involve multipart file data that might need to be specially encoded or handled differently.
- Parameters:
method (str) – The HTTP method to use in the request (GET, POST, etc.)
path (str) – A string containing the path elements for this request
params (dict) – A dictionary mapping strings to strings, to be used as the key/value pairs in the request parameters.
data – A dictionary, bytes or file-like object to send in the body.
files (dict) – A dictionary of ‘name’ => file-like-objects for multipart encoding upload.
json (dict) – A JSON object to send in the request body.
headers (dict) – If present, a dictionary of headers to encode in the request.
json_resp (bool) –
Whether the response should be parsed as JSON. If False, the raw response object is returned. To get the raw binary content of the response, use the
content
attribute of the return value, e.g.resp = client.get('my/endpoint', json_resp=False) print(resp.content) # Raw binary content print(resp.headers) # Dict of headers
retry_statuses – a sequence of HTTP response status codes to retry in addition to
dandi.consts.RETRY_STATUSES
retry_if – an optional predicate applied to a failed HTTP response to test whether to retry
- class dandi.dandiapi.DandiAPIClient(api_url: str | None = None, token: str | None = None, dandi_instance: DandiInstance | None = None)[source]#
Bases:
RESTFullAPIClient
A client for interacting with a Dandi Archive server
- authenticate(token: str, save_to_keyring: bool = False) None [source]#
Set the authentication token/API key used by the
DandiAPIClient
. Before setting the token, a test request to/auth/token
is made to check the token’s validity; if it fails, arequests.HTTPError
is raised.If
save_to_keyring
is true, then (after querying/auth/token
but before setting the API key used by the client), the token is saved in the user’s keyring at the same location as used bydandi_authenticate()
.Changed in version 0.53.0:
save_to_keyring
added
- check_schema_version(schema_version: str | None = None) None [source]#
Confirms that the server is using the same version of the Dandi schema as the client. If it is not, a
SchemaVersionError
is raised.- Parameters:
schema_version – the schema version to confirm that the server uses; if not set, the schema version for the installed
dandischema
library is used
- create_dandiset(name: str, metadata: dict[str, Any], *, embargo: bool = False) RemoteDandiset [source]#
Creates a Dandiset with the given name & metadata. If
embargo
isTrue
, the resulting Dandiset will be embargoed.Changed in version 0.61.0:
embargo
argument added
- dandi_authenticate() None [source]#
Acquire and set the authentication token/API key used by the
DandiAPIClient
. If theDANDI_API_KEY
environment variable is set, its value is used as the token. Otherwise, the token is looked up in the user’s keyring under the service “dandi-api-INSTANCE_NAME
” [1] and username “key
”. If no token is found there, the user is prompted for the token, and, if it proves to be valid, it is stored in the user’s keyring.
- classmethod for_dandi_instance(instance: str | DandiInstance, token: str | None = None, authenticate: bool = False) DandiAPIClient [source]#
Construct a client instance for the server identified by
instance
(either the name of a registered Dandi Archive instance or aDandiInstance
instance) and an optional authentication token/API key. If no token is supplied andauthenticate
is true,dandi_authenticate()
is called on the instance before returning it.
- get_asset(asset_id: str) BaseRemoteAsset [source]#
Fetch the asset with the given asset ID. If the given asset does not exist, a
NotFoundError
is raised.The returned object will not have any information about the Dandiset associated with the asset; for that, the
RemoteDandiset.get_asset()
method must be used instead.
- get_dandiset(dandiset_id: str, version_id: str | None = None, lazy: bool = True) RemoteDandiset [source]#
Fetches the Dandiset with the given
dandiset_id
. Ifversion_id
is not specified, theRemoteDandiset
’s version is set to the most recent published version if there is one, otherwise to the draft version.If
lazy
is true, no requests are actually made until any data is requested from theRemoteDandiset
.
- get_dandisets(*, draft: bool | None = None, embargoed: bool | None = None, empty: bool | None = None, mine: bool | None = None, order: str | None = None, search: str | None = None) Iterator[RemoteDandiset] [source]#
Returns a generator of all Dandisets on the server. For each Dandiset, the
RemoteDandiset
’s version is set to the most recent published version if there is one, otherwise to the draft version.Changed in version 0.61.0:
draft
,embargoed
,empty
,mine
,order
, andsearch
parameters added- Parameters:
draft – If true, Dandisets that have only draft versions (i.e., that haven’t yet been published) will be included in the results (default true)
embargoed – If true, embargoed Dandisets will be included in the results (default false)
empty – If true, empty Dandisets will be included in the results (default true)
mine – If true, only Dandisets owned by the authenticated user will be retrieved (default false)
order – The field to sort the results by. The accepted field names are
"id"
,"name"
,"modified"
, and"size"
. Prepend a hyphen to the field name to reverse the sort order.search – A search string to filter the returned Dandisets by. The string is searched for in the metadata of Dandiset versions.
- page_size: int | None#
Default number of items to request per page when paginating (
None
means to use the server’s default)
- page_workers: int#
How many pages to fetch at once when parallelizing pagination
Dandisets#
- class dandi.dandiapi.RemoteDandiset[source]#
Representation of a Dandiset (as of a certain version) retrieved from the API.
Stringifying a
RemoteDandiset
returns a string of the form"server_id:dandiset_id/version_id"
.This class should not be instantiated by end-users directly. Instead, instances should be retrieved from the appropriate attributes & methods of
DandiAPIClient
andRemoteDandiset
.- property api_path: str#
The path (relative to the base endpoint for a Dandi Archive API) at which API requests for interacting with the Dandiset itself are made
- property api_url: str#
The URL at which API requests for interacting with the Dandiset itself are made
- client: DandiAPIClient#
The
DandiAPIClient
instance that returned thisRemoteDandiset
and which the latter will use for API requests
- property contact_person: str#
The name of the registered contact person for the Dandiset
- property created: datetime#
The timestamp at which the Dandiset was created
- delete() None [source]#
Delete the Dandiset from the server. Any further access of the instance’s data attributes afterwards will result in a
NotFoundError
.
- download_directory(assets_dirpath: str, dirpath: str | Path, chunk_size: int = 8388608) None [source]#
Download all assets under the virtual directory
assets_dirpath
to the directorydirpath
. Downloads are synchronous.
- property embargo_status: EmbargoStatus#
The current embargo status for the Dandiset
- for_version(version_id: str | Version) RemoteDandiset [source]#
Returns a copy of the
RemoteDandiset
with theversion
attribute set to the givenVersion
object or theVersion
with the given version ID. If a version ID given and the version does not exist, aNotFoundError
is raised.
- classmethod from_data(client: DandiAPIClient, data: dict[str, Any]) RemoteDandiset [source]#
Construct a
RemoteDandiset
instance from aDandiAPIClient
and adict
of raw string fields in the same format as returned by the API. If the"most_recent_published_version"
field is set, that is used as the Dandiset’s version; otherwise,"draft_version"
is used.This is a low-level method that non-developers would normally only use when acquiring data using means outside of this library.
- get_asset(asset_id: str) RemoteAsset [source]#
Fetch the asset in this version of the Dandiset with the given asset ID. If the given asset does not exist, a
NotFoundError
is raised.
- get_asset_by_path(path: str) RemoteAsset [source]#
Fetch the asset in this version of the Dandiset whose
path
equalspath
. If the given asset does not exist, aNotFoundError
is raised.
- get_assets(order: str | None = None) Iterator[RemoteAsset] [source]#
Returns an iterator of all assets in this version of the Dandiset.
Assets can be sorted by a given field by passing the name of that field as the
order
parameter. The accepted field names are"created"
,"modified"
, and"path"
. Prepend a hyphen to the field name to reverse the sort order.
- get_assets_by_glob(pattern: str, order: str | None = None) Iterator[RemoteAsset] [source]#
New in version 0.44.0.
Returns an iterator of all assets in this version of the Dandiset whose
path
attributes match the glob patternpattern
Assets can be sorted by a given field by passing the name of that field as the
order
parameter. The accepted field names are"created"
,"modified"
, and"path"
. Prepend a hyphen to the field name to reverse the sort order.
- get_assets_with_path_prefix(path: str, order: str | None = None) Iterator[RemoteAsset] [source]#
Returns an iterator of all assets in this version of the Dandiset whose
path
attributes start withpath
Assets can be sorted by a given field by passing the name of that field as the
order
parameter. The accepted field names are"created"
,"modified"
, and"path"
. Prepend a hyphen to the field name to reverse the sort order.
- get_metadata() Dandiset [source]#
Fetch the metadata for this version of the Dandiset as a
dandischema.models.Dandiset
instanceNote
Only published Dandiset versions can be expected to have valid metadata. Consider using
get_raw_metadata()
instead in order to fetch unstructured, possibly-invalid metadata.
- get_raw_metadata() dict[str, Any] [source]#
Fetch the metadata for this version of the Dandiset as an unprocessed
dict
- get_version(version_id: str) VersionInfo [source]#
Get information about a given version of the Dandiset. If the given version does not exist, a
NotFoundError
is raised.Changed in version 0.49.0: This method now returns a
VersionInfo
instance instead of just aVersion
.
- get_versions(order: str | None = None) Iterator[Version] [source]#
Returns an iterator of all available
Version
s for the DandisetVersions can be sorted by a given field by passing the name of that field as the
order
parameter. Currently, the only accepted field name is"created"
. Prepend a hyphen to the field name to reverse the sort order.
- identifier: str#
The Dandiset identifier
- iter_upload_raw_asset(filepath: str | Path, asset_metadata: dict[str, Any], jobs: int | None = None, replace_asset: RemoteAsset | None = None) Iterator[dict] [source]#
Upload the file at
filepath
with metadataasset_metadata
to this version of the Dandiset, returning a generator of statusdict
s.Deprecated since version 0.36.0: Use the
iter_upload()
method ofLocalAsset
instances instead- Parameters:
filepath (str or PathLike) – the path to the local file to upload
asset_metadata (dict) – Metadata for the uploaded asset file. Must include a “path” field giving the forward-slash-separated path at which the uploaded file will be placed on the server.
jobs (int) – Number of threads to use for uploading; defaults to 5
replace_asset (RemoteAsset) – If set, replace the given asset, which must have the same path as the new asset
- Returns:
A generator of
dict
s containing at least a"status"
key. Upon successful upload, the lastdict
will have a status of"done"
and an"asset"
key containing the resultingRemoteAsset
.
- json_dict() dict[str, Any] [source]#
Convert to a JSONable
dict
, omitting theclient
attribute and using the same field names as in the API
- property modified: datetime#
The timestamp at which the Dandiset was last modified
- property most_recent_published_version: Version | None#
The most recent published (non-draft) version of the Dandiset, or
None
if no versions have been published
- publish(max_time: float = 120) RemoteDandiset [source]#
Publish the draft version of the Dandiset and wait at most
max_time
seconds for the publication operation to complete. If the operation does not complete in time, aValueError
is raised.Returns a copy of the
RemoteDandiset
with theversion
attribute set to the new publishedVersion
.
- refresh() None [source]#
Update the
RemoteDandiset
in-place with the latest data from the server. TheRemoteDandiset
continues to have the same version as before, but the cached version data is internally cleared and may be different upon subsequent access.
- set_metadata(metadata: Dandiset) None [source]#
Set the metadata for this version of the Dandiset to the given value
- set_raw_metadata(metadata: dict[str, Any]) None [source]#
Set the metadata for this version of the Dandiset to the given value
- upload_raw_asset(filepath: str | Path, asset_metadata: dict[str, Any], jobs: int | None = None, replace_asset: RemoteAsset | None = None) RemoteAsset [source]#
Upload the file at
filepath
with metadataasset_metadata
to this version of the Dandiset and return the resulting asset. Blocks until the upload is complete.Deprecated since version 0.36.0: Use the
upload()
method ofLocalAsset
instances instead- Parameters:
filepath (str or PathLike) – the path to the local file to upload
asset_metadata (dict) – Metadata for the uploaded asset file. Must include a “path” field giving the forward-slash-separated path at which the uploaded file will be placed on the server.
jobs (int) – Number of threads to use for uploading; defaults to 5
replace_asset (RemoteAsset) – If set, replace the given asset, which must have the same path as the new asset
- property version_api_path: str#
The path (relative to the base endpoint for a Dandi Archive API) at which API requests for interacting with the version in question of the Dandiset are made
- property version_api_url: str#
The URL at which API requests for interacting with the version in question of the Dandiset are made
- property version_id: str#
The identifier for the Dandiset version
- class dandi.dandiapi.Version[source]#
The version information for a Dandiset retrieved from the API.
Stringifying a
Version
returns its identifier.This class should not be instantiated by end-users directly. Instead, instances should be retrieved from the appropriate attributes & methods of
RemoteDandiset
.- asset_count: int#
The number of assets in the version
- created: datetime#
The timestamp at which the version was created
- identifier: str#
The version identifier
- json_dict() dict[str, Any] #
Convert to a JSONable
dict
, omitting theclient
attribute and using the same field names as in the API
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'asset_count': FieldInfo(annotation=int, required=True), 'created': FieldInfo(annotation=datetime, required=True), 'identifier': FieldInfo(annotation=str, required=True, alias='version', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'name': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'status': FieldInfo(annotation=VersionStatus, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- modified: datetime#
The timestamp at which the version was last modified
- name: str#
The name of the version
- size: int#
The total size in bytes of all assets in the version
- status: VersionStatus#
- class dandi.dandiapi.VersionInfo[source]#
Bases:
Version
New in version 0.49.0.
Version information for a Dandiset, including information about validation errors
- asset_validation_errors: List[RemoteAssetValidationError]#
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'asset_count': FieldInfo(annotation=int, required=True), 'asset_validation_errors': FieldInfo(annotation=List[dandi.dandiapi.RemoteAssetValidationError], required=True), 'created': FieldInfo(annotation=datetime, required=True), 'identifier': FieldInfo(annotation=str, required=True, alias='version', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'name': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'status': FieldInfo(annotation=VersionStatus, required=True), 'version_validation_errors': FieldInfo(annotation=List[dandi.dandiapi.RemoteValidationError], required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- version_validation_errors: List[RemoteValidationError]#
- class dandi.dandiapi.RemoteValidationError[source]#
New in version 0.49.0.
Validation error record obtained from a server. Not to be confused with
dandi.validate_types.ValidationResult
, which provides richer representation of validation errors.- field: str#
- json_dict() dict[str, Any] #
Convert to a JSONable
dict
, omitting theclient
attribute and using the same field names as in the API
- message: str#
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'field': FieldInfo(annotation=str, required=True), 'message': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
Assets#
- class dandi.dandiapi.BaseRemoteAsset[source]#
Representation of an asset retrieved from the API without associated Dandiset information.
This is an abstract class; its concrete subclasses are
BaseRemoteBlobAsset
(for assets backed by blobs) andBaseRemoteZarrAsset
(for assets backed by Zarrs).Stringifying a
BaseRemoteAsset
returns a string of the form"server_id:asset/asset_id"
.This class should not be instantiated by end-users directly. Instead, instances should be retrieved from the appropriate methods of
DandiAPIClient
.- property api_path: str#
The path (relative to the base endpoint for a Dandi Archive API) at which API requests for interacting with the asset itself are made
- property api_url: str#
The URL at which API requests for interacting with the asset itself are made
- abstract property asset_type: AssetType#
New in version 0.36.0.
The type of the asset’s underlying data
- property base_download_url: str#
The URL from which the asset can be downloaded, sans any Dandiset identifiers (cf.
RemoteAsset.download_url
)
- client: DandiAPIClient#
The
DandiAPIClient
instance that returned thisBaseRemoteAsset
and which the latter will use for API requests
- created: datetime#
The date at which the asset was created
- property digest_type: DigestType#
New in version 0.36.0.
The primary digest algorithm used by Dandi Archive for the asset, determined based on its underlying data: dandi-etag for blob resources, dandi-zarr-checksum for Zarr resources
- download(filepath: str | Path, chunk_size: int = 8388608) None [source]#
Download the asset to
filepath
. Blocks until the download is complete.- Raises:
ValueError – if the asset is not backed by a blob
- classmethod from_base_data(client: DandiAPIClient, data: dict[str, Any], metadata: dict[str, Any] | None = None) BaseRemoteAsset [source]#
Construct a
BaseRemoteAsset
instance from aDandiAPIClient
, adict
of raw data in the same format as returned by the API’s pagination endpoints, and optional raw asset metadata.This is a low-level method that non-developers would normally only use when acquiring data using means outside of this library.
- get_content_url(regex: str = '.*', follow_redirects: bool | int = False, strip_query: bool = False) str [source]#
Returns a URL for downloading the asset, found by inspecting the metadata; specifically, returns the first
contentUrl
that matchesregex
. RaisesNotFoundError
if the metadata does not contain a matching URL.If
follow_redirects
isTrue
, aHEAD
request is made to resolve any & all redirects before returning the URL. Iffollow_redirects
is an integer, at most that many redirects are followed.If
strip_query
is true, any query parameters are removed from the final URL before returning it.
- get_digest() Digest [source]#
New in version 0.36.0: Replaces the previous version of
get_digest()
, now renamed toget_raw_digest()
Retrieves the DANDI etag digest of the appropriate type for the asset: a dandi-etag digest for blob resources or a dandi-zarr-checksum for Zarr resources
- get_download_file_iter(chunk_size: int = 8388608) Callable[[int], Iterator[bytes]] [source]#
Returns a function that when called (optionally with an offset into the asset to start downloading at) returns a generator of chunks of the asset.
- Raises:
ValueError – if the asset is not backed by a blob
- get_metadata() Asset [source]#
Fetch the metadata for the asset as a
dandischema.models.Asset
instanceNote
Only assets in published Dandiset versions can be expected to have valid metadata. Consider using
get_raw_metadata()
instead in order to fetch unstructured, possibly-invalid metadata.
- get_raw_digest(digest_type: str | models.DigestType | None = None) str [source]#
Retrieves the value of the given type of digest from the asset’s metadata. Raises
NotFoundError
if there is no entry for the given digest type.If no digest type is specified, the same type as used by
get_digest()
is returned.Changed in version 0.36.0: Renamed from
get_digest()
toget_raw_digest()
- identifier: str#
The asset identifier
- json_dict() dict[str, Any] #
Convert to a JSONable
dict
, omitting theclient
attribute and using the same field names as in the API
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- model_post_init(__context: Any) None #
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
- Args:
self: The BaseModel instance. __context: The context.
- modified: datetime#
The date at which the asset was last modified
- path: str#
The asset’s (forward-slash-separated) path
- size: int#
The size of the asset in bytes
- class dandi.dandiapi.BaseRemoteBlobAsset[source]#
Bases:
BaseRemoteAsset
New in version 0.36.0.
A
BaseRemoteAsset
whose actual data is a blob resource- as_readable() RemoteReadableAsset [source]#
New in version 0.50.0.
Returns a
Readable
instance that can be used to obtain a file-like object for reading bytes directly from the asset on the server
- blob: str#
The ID of the underlying blob resource
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'blob': FieldInfo(annotation=str, required=True), 'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- class dandi.dandiapi.AssetType(value)[source]#
New in version 0.36.0.
An enum for the different kinds of resources that an asset’s actual data can be
- BLOB = 1#
- ZARR = 2#
- class dandi.dandiapi.RemoteAsset[source]#
Bases:
BaseRemoteAsset
Subclass of
BaseRemoteAsset
that includes information about the Dandiset to which the asset belongs.This is an abstract class; its concrete subclasses are
RemoteBlobAsset
(for assets backed by blobs) andRemoteZarrAsset
(for assets backed by Zarrs).This class should not be instantiated by end-users directly. Instead, instances should be retrieved from the appropriate attributes & methods of
RemoteDandiset
.- property api_path: str#
The path (relative to the base endpoint for a Dandi Archive API) at which API requests for interacting with the asset itself are made
- property api_url: str#
The URL at which API requests for interacting with the asset itself are made
- dandiset_id: str#
The identifier for the Dandiset to which the asset belongs
- property download_url: str#
The URL from which the asset can be downloaded, including Dandiset identifiers (cf.
BaseRemoteAsset.base_download_url
)
- classmethod from_data(dandiset: RemoteDandiset, data: dict[str, Any], metadata: dict[str, Any] | None = None) RemoteAsset [source]#
Construct a
RemoteAsset
instance from aRemoteDandiset
, adict
of raw data in the same format as returned by the API’s pagination endpoints, and optional raw asset metadata.This is a low-level method that non-developers would normally only use when acquiring data using means outside of this library.
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'dandiset_id': FieldInfo(annotation=str, required=True, exclude=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'version_id': FieldInfo(annotation=str, required=True, exclude=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- rename(dest: str) None [source]#
New in version 0.41.0.
Change the path of the asset on the server to the given value and update the
RemoteAsset
in place. If another asset already exists at the given path, arequests.HTTPError
is raised.
- set_metadata(metadata: Asset) None [source]#
Set the metadata for the asset to the given value and update the
RemoteAsset
in place.
- abstract set_raw_metadata(metadata: dict[str, Any]) None [source]#
Set the metadata for the asset on the server to the given value and update the
RemoteAsset
in place.
- version_id: str#
The identifier for the version of the Dandiset to which the asset belongs
- class dandi.dandiapi.RemoteBlobAsset[source]#
Bases:
RemoteAsset
,BaseRemoteBlobAsset
New in version 0.36.0.
A
RemoteAsset
whose actual data is a blob resource- client: DandiAPIClient#
The
DandiAPIClient
instance that returned thisBaseRemoteAsset
and which the latter will use for API requests
- created: datetime#
The date at which the asset was created
- dandiset_id: str#
The identifier for the Dandiset to which the asset belongs
- identifier: str#
The asset identifier
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'blob': FieldInfo(annotation=str, required=True), 'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'dandiset_id': FieldInfo(annotation=str, required=True, exclude=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'version_id': FieldInfo(annotation=str, required=True, exclude=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- modified: datetime#
The date at which the asset was last modified
- path: str#
The asset’s (forward-slash-separated) path
- set_raw_metadata(metadata: dict[str, Any]) None [source]#
Set the metadata for the asset on the server to the given value and update the
RemoteBlobAsset
in place.
- size: int#
The size of the asset in bytes
- version_id: str#
The identifier for the version of the Dandiset to which the asset belongs
Zarr Assets#
- class dandi.dandiapi.BaseRemoteZarrAsset[source]#
Bases:
BaseRemoteAsset
New in version 0.36.0.
A
BaseRemoteAsset
whose actual data is a Zarr resource- get_entry_by_path(path: str) RemoteZarrEntry [source]#
Fetch the entry in this Zarr whose
path
equalspath
. If the given entry does not exist, aNotFoundError
is raised.
- iterfiles(prefix: str | None = None) Iterator[RemoteZarrEntry] [source]#
Returns a generator of all
RemoteZarrEntry
s within the Zarr, optionally limited to those whose path starts with the given prefix
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'zarr': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- rmfiles(files: Iterable[RemoteZarrEntry], reingest: bool = True) None [source]#
Delete one or more files from the Zarr.
If
reingest
is true, after performing the deletion, the client triggers a recalculation of the Zarr’s checksum and waits for it to complete.
- zarr: str#
The ID of the underlying Zarr resource
- class dandi.dandiapi.RemoteZarrAsset[source]#
Bases:
RemoteAsset
,BaseRemoteZarrAsset
New in version 0.36.0.
A
RemoteAsset
whose actual data is a Zarr resource- client: DandiAPIClient#
The
DandiAPIClient
instance that returned thisBaseRemoteAsset
and which the latter will use for API requests
- created: datetime#
The date at which the asset was created
- dandiset_id: str#
The identifier for the Dandiset to which the asset belongs
- identifier: str#
The asset identifier
- model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#
A dictionary of computed field names and their corresponding
ComputedFieldInfo
objects.
- model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'populate_by_name': True}#
Configuration for the model, should be a dictionary conforming to [
ConfigDict
][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'client': FieldInfo(annotation=DandiAPIClient, required=True, exclude=True), 'created': FieldInfo(annotation=datetime, required=True), 'dandiset_id': FieldInfo(annotation=str, required=True, exclude=True), 'identifier': FieldInfo(annotation=str, required=True, alias='asset_id', alias_priority=2), 'modified': FieldInfo(annotation=datetime, required=True), 'path': FieldInfo(annotation=str, required=True), 'size': FieldInfo(annotation=int, required=True), 'version_id': FieldInfo(annotation=str, required=True, exclude=True), 'zarr': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [
FieldInfo
][pydantic.fields.FieldInfo].This replaces
Model.__fields__
from Pydantic V1.
- modified: datetime#
The date at which the asset was last modified
- path: str#
The asset’s (forward-slash-separated) path
- set_raw_metadata(metadata: dict[str, Any]) None [source]#
Set the metadata for the asset on the server to the given value and update the
RemoteZarrAsset
in place.
- size: int#
The size of the asset in bytes
- version_id: str#
The identifier for the version of the Dandiset to which the asset belongs
- class dandi.dandiapi.RemoteZarrEntry[source]#
Bases:
object
New in version 0.36.0.
A file within a
RemoteZarrAsset
Changed in version 0.48.0:
No longer represents directories
No longer implements
BasePath
- client: DandiAPIClient#
The
DandiAPIClient
instance used for API requests
- get_download_file_iter(chunk_size: int = 8388608) Callable[[int], Iterator[bytes]] [source]#
Returns a function that when called (optionally with an offset into the file to start downloading at) returns a generator of chunks of the file
- modified: datetime#
The timestamp at which the entry was last modified
- property name: str#
The basename of the path object
- parts: tuple[str, ...]#
The path components of the entry
- size: int#
The entry’s size in bytes
- property stem: str#
The basename without its final file extension, if any
- property suffix: str#
The final file extension of the basename, if any
- property suffixes: list[str]#
A list of the basename’s file extensions
- zarr_id: str#
The ID of the Zarr backing the asset