[Remi-Gau]

The size of the datasets (size, from datalad status --annex all) includes the content of their derivatives and sourcedata folders, which can be large and are not standardized.

New field: raw_size

• The size in bytes of the annexed files, without the derivatives, sourcedata (and source) folders at the root of the dataset.
• From the size in the git-annex keys of the files (git annex find --branch=HEAD --format='${bytesize}\t${file}\n'): fast (no datalad status), even if the content of the files is not downloaded.
• The files of the last commit, whatever the state of the working tree: for example the local copy of ds004572 had all its files staged as deleted, and the working tree gave 0 bytes instead of 46.8 GB.
• Read when indexing a dataset, and can be updated with bids_inventory update --fields raw_size (both sources).

In the dashboard

wrangle_data gives size (and mean_size, per participant) in bytes from raw_size when known, otherwise from the size reported by datalad. The sizes of datalad are now read in binary units (2.2 GB is 2.2 × 1024³ bytes, they were read as 2.2 × 10⁹).

Listings

• OpenNeuro: raw_size for 1,920 of the 1,929 datasets: 130.7 TB instead of 182.8 TB. Not for 6 datasets not installed locally yet, and 3 whose local copy is not initialized with git-annex (ds004186, ds004935, ds006391): they keep the size of datalad.
• Other datasets: all 131 (3.6 TB instead of 3.7 TB).
• Some datasets are now larger than their previous size: it had been read from an older copy (for example ds006182: 6.7 GB in the superdataset from May 2025, 30.6 GiB in its copy from February 2026). Datasets with only derivatives (ds006253) have a raw_size of 0.

🤖 Generated with Claude Code