feat(provenance): add PROV-O RDF export for BrainKB ingestion - #386
Open
dhritimandas wants to merge 7 commits into
Open
feat(provenance): add PROV-O RDF export for BrainKB ingestion#386dhritimandas wants to merge 7 commits into
dhritimandas wants to merge 7 commits into
Conversation
Add versioned dataset contract with DataLad-aware preflight validation: - DataSpec dataclass with JSON serialization - validate() with symlink-aware file checks, spacing/orientation/label validation - inspect_entry() for NIfTI/Zarr metadata inspection - CLI commands: validate, inspect, --skip-validate on predict/convert-to-zarr - 33 unit tests covering all branches
No structured provenance graph existed for a training run -- croissant.json records model/dataset facts but no prov:Activity linking them. This adds `nobrainer provenance export`, reading a saved model bundle (and optionally a DataSpec manifest for image+label dataset provenance) into a PROV-O graph: the run as prov:Activity, used dataset entity, generated model entity, and associated software/caller agents. Every IRI is content-derived (sha256 for files/models, stable digests for runs/datasets/agents) -- no blank nodes, no timestamps or UUIDs in identity, so re-export of the same input is byte-identical regardless of its path on disk. Emits DOMAIN provenance only: named-graph registration and POSTing to BrainKB's ingestion API are separate steps against BrainKB's own API, enforced at runtime (not just documented) by rejecting a second Activity, any non-SoftwareAgent floating free of the run, or any brainkb-named term. rdflib is the one new dependency, gated behind a new [provenance] extra (it was already present transitively via mlcroissant, but undeclared).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What: This adds a command. The command is
nobrainer provenance export.It reads a saved model bundle. It emits a PROV-O graph in Turtle or JSON-LD.
Why: No structured provenance graph existed for a training run.
croissant.jsonrecords model and dataset facts, but nothing links them asan activity. BrainKB needs that graph to ingest nobrainer runs.
The graph
The run is a
prov:Activity. Itprov:useda dataset entity andprov:generateda model entity. Both link to one or moreprov:wasAssociatedWithagents (always thenobrainer/pytorchsoftwareagents; optionally a caller-supplied human/org agent via
--agent, neverinvented by this module on its own).
Determinism
Every IRI is content-derived: a file/model's own sha256, or a stable hash
of a canonical-JSON payload for the run/dataset/agent nodes. No timestamps,
no random UUIDs, no blank nodes anywhere. Re-exporting the same input from
a different location on disk produces a byte-identical graph -- verified
directly, not just asserted (
test_run_iri_unchanged_after_directory_move,test_repeated_export_is_byte_identical).DOMAIN provenance only -- enforced, not just documented
This module emits facts about what nobrainer itself did. It does not
emit BrainKB ingestion-activity triples (e.g. "this graph was loaded into
BrainKB at time T"). Named-graph registration and POSTing this output to
BrainKB's own ingestion API are separate steps the caller performs against
BrainKB's API. A runtime check (
_assert_domain_only, called on everybuild_graph()) rejects: a secondprov:Activity, any agent notassociated with the run, forbidden PROV predicates
(
wasInformedBy/wasStartedBy/wasEndedBy/qualifiedAssociation/hadPlan), and any term containing"brainkb". A positive-control testinjects a second activity and confirms the check actually fires.
Inputs
--bundle DIR(required): aSegmentation.save()directory(
model.pth+croissant.json). Read with plainjson.load-- not vianobrainer.processing.croissant, since importing that module pulls intorch through
processing/__init__.py(verified). This module isstdlib + rdflib only.
--dataspec PATH.json(optional): aDataSpecmanifest(
nobrainer/data/spec.py). When given, dataset provenance includes bothimage and label checksums. When omitted, falls back to croissant's
source_datasets, which only ever records image paths.--agent,--base-iri,--strict,--format {turtle,json-ld}.Edge cases handled explicitly, not silently
final_loss/best_lossis omitted (nb:lossStatus "non-finite") rather than serialized: rdflib emits a bareNaNinJSON-LD, which is not valid JSON (verified).
model_args/optimizer.argsvalue that landed in croissant.json as aPython
repr()memory address (write_model_croissantusesdefault=str) is elided rather than hashed -- it would otherwise makerun IRIs nondeterministic across machines.
block_shape/Nonefields emit no triple (absence, not"").unaddressed
x-nosha/IRI, recorded vianb:checksumStatus.write_model_croissantandwrite_checkpoint_croissantdisagree onmodel_architecturevocabulary (registry name vs. torch class name);the disagreement is recorded (
nb:architectureVocabulary), not guessed.The one new dependency
rdflibwas already present transitively (viamlcroissant) butundeclared. Now
[project.optional-dependencies] provenance = ["rdflib >= 7.0"], added toall.Known gap, flagged rather than papered over
training_datein croissant.json is the model-save timestamp, not a runstart/end. It is emitted as
prov:generatedAtTimeon the model (whereit's literally true) and as a neutral
nb:metadataRecordedAton the run-- never as
prov:endedAtTime, which would misrepresent it. The graphgenuinely cannot express training duration until
write_model_croissantrecords real start/end times.
Test plan
uv run pytest nobrainer/tests/unit/test_rdf_export.py -q-- 52passed (hashing primitives, graph shape,
--agent,--dataspec,edge cases, the DOMAIN-only boundary with a positive control,
turtle/JSON-LD equivalence, determinism, no-torch-import isolation,
CLI subprocess tests).
uv run pytest nobrainer/tests/unit/ -m "not gpu" -q-- 446 passed,no regressions.
pre-commit run --all-filesclean.re-parse cleanly with a fresh
rdflib.Graph().parse()and producethe same triple count (81 == 81).