Skip to content

Fix schema-2 result storage provenance and provide safe repair - #44

Merged
TorecLuik merged 1 commit into
masterfrom
fix/schema2-storage-provenance
Oct 20, 2026
Merged

TorecLuik merged 1 commit into
masterfrom
fix/schema2-storage-provenance

Conversation

@TorecLuik

@TorecLuik TorecLuik commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Successful shallow imports lost storage provenance because the reader expected top-level images and nested source records. Parse the actual schema-2 graph/binding contract with ShallowManifest, select the correct Image node, and preserve workflow, managed-path, receipt and ambiguity checks. Full-Zarr handling remains independent of shallow feature flags.

Partial metadata now produces an explicit WARNING in the import script's Message while keeping imported data available. Storage evidence is persisted before rendering workflow maps.

Add an administrator-only, dry-run-by-default backfill that reads existing managed evidence and records it once, refusing conflicting history. Expose an explicit storage-repair option through the existing metadata persistence adapter, retaining its backups and shared-annotation protections. The README documents both steps. Pixel arrays are never read or rewritten by repair.

Closes #41.

Validation: 293 passed, 4 existing skips across the complete separate harness, plus script compilation and git diff --check. Initial schema-2 regression run: 23 failed / 6 guards passed; the additional repair-adapter and conflicting-evidence tests also failed before their fixes. Real schema-2 fixtures cover Image/Plate bindings, both interchange profiles, invalid manifests/references, warnings, persistence order and idempotent repair. Live OMERO/ACC acceptance remains to be run after deployment.

Dependencies and merge order:

Kept in draft until the canonical harness prerequisite is merged. No tests are included in the deployable branch; no release or data repair was performed.

@TorecLuik
TorecLuik marked this pull request as ready for review October 20, 2026 13:16
@TorecLuik
TorecLuik merged commit 14d2cd1 into master Oct 20, 2026
1 of 2 checks passed
@TorecLuik
TorecLuik deleted the fix/schema2-storage-provenance branch October 20, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema-2 shallow imports lose storage provenance with KeyError('images'); still present in beta.14

1 participant