Repository navigation
Preserve failed results and recover imports with automatic legacy fallback - #50
Merged
Merged
Conversation
This was referenced Oct 27, 2026
This was referenced Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Importer failures and polling timeouts now fail finalization and preserve staged results, archives and remote recovery material. Cleanup runs only after success. On compatible storage, finalization saves one small plan per new workflow-result import; retries reuse its destination, rename mapping, options and order UUIDs instead of registering duplicates.
Recovery is automatic, with no new feature flag or historical-data migration. Before preparing a destination or queueing imports, the script acquires a filesystem lock and probes durable writes using disposable metadata in the same directory: file synchronization, replacement of an existing file and POSIX directory synchronization. Explicit unsupported-operation errors, including EINVAL from synchronization, use legacy operations with a warning. This checks API support; it does not certify arbitrary filesystems' cross-host locking or power-loss guarantees.
Busy locks, permission failures, disk-full errors, storage outages and failures after recovery begins stop safely. Existing saved plans cannot be bypassed. Failed/partial registrations, interrupted preparation and older staged results with existing orders require review. A competing finalizer cannot overwrite the active import status. ROI/attachment postprocessing remain outside the image-registration reuse protection.
Existing feature flags remain authoritative. Automatic order recovery covers full Zarr/TIFF imports independently of detached execution and shallowing. Slurm Init now acquires the remote shallower image only when importer integration, shallow storage and the remote preference are enabled; previously it could acquire that image with shallow storage disabled. Reconstruction of existing shallow inputs remains available.
Validation: red/green regressions for automatic recovery, filesystem fallback and disabled-feature initialization. Full Windows harness 386 passed, 7 skipped; unchanged master 330 passed, 63 skipped. The preceding recovery implementation also passed all 51 recovery cases on Linux, real importer SQL writer/isolated SQLite acknowledgement-loss and timeout scenarios, and cross-container shared-lock/process-exit checks. Compilation and diff checks pass. Full Slurm/HPC acceptance remains pending.
Recovery harness #52 is merged. Merge the additional flag regression tests into test-suite and rerun this PR to include them in CI; they already pass locally. Pair with importer #31. Core docs and NL-BIOMERO #52 describe the boundaries. Coordinated component publication, dependency-pin alignment and assembled-stack acceptance remain pending.