Skip to content

Preserve failed results and recover imports with automatic legacy fallback - #50

Merged
TorecLuik merged 4 commits into
masterfrom
fix/result-finalization-recovery
Oct 27, 2026
Merged

TorecLuik merged 4 commits into
masterfrom
fix/result-finalization-recovery

Conversation

@TorecLuik

@TorecLuik TorecLuik commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Importer failures and polling timeouts now fail finalization and preserve staged results, archives and remote recovery material. Cleanup runs only after success. On compatible storage, finalization saves one small plan per new workflow-result import; retries reuse its destination, rename mapping, options and order UUIDs instead of registering duplicates.

Recovery is automatic, with no new feature flag or historical-data migration. Before preparing a destination or queueing imports, the script acquires a filesystem lock and probes durable writes using disposable metadata in the same directory: file synchronization, replacement of an existing file and POSIX directory synchronization. Explicit unsupported-operation errors, including EINVAL from synchronization, use legacy operations with a warning. This checks API support; it does not certify arbitrary filesystems' cross-host locking or power-loss guarantees.

Busy locks, permission failures, disk-full errors, storage outages and failures after recovery begins stop safely. Existing saved plans cannot be bypassed. Failed/partial registrations, interrupted preparation and older staged results with existing orders require review. A competing finalizer cannot overwrite the active import status. ROI/attachment postprocessing remain outside the image-registration reuse protection.

Existing feature flags remain authoritative. Automatic order recovery covers full Zarr/TIFF imports independently of detached execution and shallowing. Slurm Init now acquires the remote shallower image only when importer integration, shallow storage and the remote preference are enabled; previously it could acquire that image with shallow storage disabled. Reconstruction of existing shallow inputs remains available.

Validation: red/green regressions for automatic recovery, filesystem fallback and disabled-feature initialization. Full Windows harness 386 passed, 7 skipped; unchanged master 330 passed, 63 skipped. The preceding recovery implementation also passed all 51 recovery cases on Linux, real importer SQL writer/isolated SQLite acknowledgement-loss and timeout scenarios, and cross-container shared-lock/process-exit checks. Compilation and diff checks pass. Full Slurm/HPC acceptance remains pending.

Recovery harness #52 is merged. Merge the additional flag regression tests into test-suite and rerun this PR to include them in CI; they already pass locally. Pair with importer #31. Core docs and NL-BIOMERO #52 describe the boundaries. Coordinated component publication, dependency-pin alignment and assembled-stack acceptance remain pending.

@TorecLuik TorecLuik changed the title Preserve failed results and resume durable import orders without duplicates Preserve failed results and add compatible opt-in import recovery Oct 27, 2026
@TorecLuik TorecLuik changed the title Preserve failed results and add compatible opt-in import recovery Preserve failed results and recover imports with automatic legacy fallback Oct 27, 2026
@TorecLuik
TorecLuik merged commit f46199f into master Oct 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant