Skip to content

Repository files navigation

template_data_descriptor

template_data_descriptor is a public exemplar for FAIR-style data descriptor papers and dataset release packets. It treats the dataset, schema, provenance, licensing, and validation report as the research object under test.

Run via the template monorepo from the repository root with uv run python scripts/pipeline/stage_01_test.py --project templates/template_data_descriptor --project-only. Copy docs/manuscript/config.yaml.example to docs/manuscript/config.yaml in forks, preserve template integrity, and keep output artifacts regenerated from source.

When to use this template

Use this template when the contribution is a published dataset or data paper: a schema contract, file inventory, data dictionary, provenance chain, license boundary, quality checks, and machine-readable descriptor must all stay consistent before publication.

Publication and rendering

Data Descriptor Template: Schema, Provenance, and Release Readiness · v0.1.0 · Daniel Ari Friedman

Concept DOI: 10.5281/zenodo.21298883 | Version DOI: 10.5281/zenodo.21298884 | Repository: docxology/template_data_descriptor

Publishing surface — 20 platforms, 2 published:

Platform Tier Status Reference Credentials
zenodo first-class ✅ published 10.5281/zenodo.21298883 ZENODO_API_TOKEN
github first-class ✅ published docxology/template_data_descriptor GITHUB_TOKEN
arxiv first-class ⚪ available
pypi first-class ⚪ available PYPI_TOKEN, TESTPYPI_TOKEN
ipfs_pinata first-class ⚪ available PINATA_JWT
ipfs_web3storage first-class ⚪ available WEB3_STORAGE_TOKEN
software_heritage first-class ⚪ available
github_pages first-class ⚪ available docxology/template_data_descriptor GITHUB_TOKEN
cloudflare_pages first-class ⚪ available CLOUDFLARE_API_TOKEN
netlify first-class ⚪ available NETLIFY_AUTH_TOKEN
huggingface_hub first-class ⚪ available HUGGINGFACE_TOKEN, HF_TOKEN
osf first-class ⚪ available OSF_TOKEN
amazon_kdp documented 🟡 planned AMAZON_KDP_EMAIL, AMAZON_KDP_PASSWORD
google_play_books documented 🟡 planned GOOGLE_PLAY_BOOKS_SERVICE_ACCOUNT_JSON
gumroad documented 🟡 planned GUMROAD_ACCESS_TOKEN
leanpub documented 🟡 planned LEANPUB_API_KEY
lulu documented 🟡 planned LULU_CLIENT_KEY, LULU_CLIENT_SECRET
draft2digital documented 🟡 planned DRAFT2DIGITAL_API_TOKEN
stripe documented 🟡 planned STRIPE_SECRET_KEY, STRIPE_PUBLISHABLE_KEY
ingramspark documented 🟡 planned INGRAMSPARK_CLIENT_ID, INGRAMSPARK_CLIENT_SECRET

Keywords: data descriptor, FAIR data, provenance, schema validation.

Status legend: ✅ published (durable identifier recorded in config.yaml) · 🔵 reserved (identifier reserved but not yet registered by final publication) · ⚪ available (adapter implemented and locally verifiable) · 🟡 planned. This block is generated — edit docs/manuscript/config.yaml, then regenerate with uv run python -m infrastructure.publishing.status_report --project <path> --write.

The canonical renderer is https://github.com/docxology/template with --project templates/template_data_descriptor. The project ships a minimal manuscript shell and source-owned validation library; generated PDFs, HTML, and release artifacts belong under output/ and are regenerated by the monorepo pipeline.

Configuration

Primary configuration lives in docs/manuscript/config.yaml; forkable defaults live in docs/manuscript/config.yaml.example. Dataset-facing metadata lives in data/example_descriptor.json, and the public synthetic fixture bytes live under data/fixtures/.

Figures and manuscript

The manuscript embeds five figures generated deterministically from the descriptor and fixture bytes by the thin script scripts/generate_figures.py (schema data dictionary, file inventory, provenance flow, quality gate, and descriptor↔file checksum verification). All computation lives in the tested src/data_descriptor/ package (descriptor.py, figures.py, rendering.py, verification.py); the script only orchestrates rendering and prints paths. Figures are written to and embedded from docs/manuscript/figures/. Regenerate with:

uv run python projects/templates/template_data_descriptor/scripts/generate_figures.py

Tests

Run:

uv run pytest projects/templates/template_data_descriptor/tests --cov=projects/templates/template_data_descriptor/src --cov-fail-under=90

Outputs and validation

The core validation output is a descriptor report containing schema fingerprint, field counts, file inventory checks, license status, field-level constraint checks, and quality-gate findings. Byte-level verification (verify_descriptor_files) recomputes each declared file's sha256 checksum and row count and reconciles them against the descriptor; JSON is supported, while other media require an explicit justification. The package also builds a deterministic metadata-only release manifest with file checksums, schema fingerprint, provenance steps, and unit/bounds/enumeration summaries for pre-publication review, written under output/reports/ by scripts/generate_release_artifacts.py. A publication receipt additionally requires a real repository identity, matching descriptor digest, and explicit owner attestation. Stage 04 validation checks the rendered manuscript and publication boundary once outputs exist.

Publication and boundaries

This exemplar is intentionally about publishing data responsibly, not about claiming scientific effects from the data. Claims should stay limited to dataset structure, provenance, quality, and release-readiness evidence.

Fork guidance

Use scripts/audit/copy_exemplar.py to fork this template cleanly. Replace placeholder descriptors, keep domain_profile.yaml and experiment_plan.yaml aligned with your dataset, and extend field constraints before publishing real data.

About

Scaffold for publishing a dataset with a machine-readable data contract: JSON descriptors drive schema fingerprints, field-level constraint checks, SHA-256 file verification and quality gates, emitting data-dictionary and provenance-flow figures plus a FAIR release manifest with durable Zenodo identifiers.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages