Preserve native CI failure evidence and verify archived diagnostics

notes/native_ci_recovery_2026-10-08/README.md

@@ -0,0 +1,70 @@
+# Retained synthetic native CI evidence
+
+This package preserves and checks the saved artifacts from [PR #4659, run 37759818157](https://github.com/bridge2ai/data-sheets-schema/actions/runs/37759818157). It does not run native code, replay the cases, or establish whole-path acceptance. The neutral case stopped on a stream-change refusal; the omission case reached the existing deadline. Both failed outcomes are retained.
+
+The tools address [#4679](https://github.com/bridge2ai/data-sheets-schema/issues/4679). The early stream failure is tracked separately in [#4680](https://github.com/bridge2ai/data-sheets-schema/issues/4680), with diagnostic work in [#4682](https://github.com/bridge2ai/data-sheets-schema/issues/4682); [#4537](https://github.com/bridge2ai/data-sheets-schema/issues/4537) and [#4354](https://github.com/bridge2ai/data-sheets-schema/issues/4354) remain open. No native production modules, acceptance assertions, workflow scheduling, or 900/180/120-second bounds are changed by this evidence package.
+
+## Identity and retained bytes
+
+The tested PR head is `aa845f2e083185707feab0ee8b090d6d9c9f52ef`. GitHub tested merge `cfe68ee650e3de0f775bd476f055f25c29eb7a72`, tree `8bdb68b58a410756871bf5b99cba90fbc8d6581b`. This historical identity is independent of the main branch containing these notes.
+
+`evidence/run-37759818157/` retains each recovered native case ZIP, its GitHub artifact metadata and offline verification report, matching JUnit ZIPs, and literal inspection summaries for the two failed cases. Small archives are committed to preserve the evidence after GitHub artifact expiration. ZIP/tar contents are inspected as bytes and are never extracted or imported. GitHub runner paths within original artifacts are historical declarations, not local input paths.
+
+`expected.json` declares the tested merge, tree, PR head and eight selected source-file pins. `source-pin-provenance.json` records their Git blob IDs and retained source bytes under `source-bytes/`. These bytes were downloaded at the explicit merge and compared with `git show` of that same fetched commit; tree and PR-parent linkage were checked independently of the case manifests. Retained source files have `.txt` names and are not loaded as code. This is a selected source identity, not the complete runtime import closure or authenticated execution.
+
+## Saved outcomes
+
+| Case | Shard / worker | JUnit duration | Saved first stop |
+| --- | --- | ---: | --- |
+| `test_actual_public_correction_three_workers_and_saved_completion` | 1 / `gw3` | 253.860 s | `native shared observation failed: ValueError: live native stream changed without appending bytes` |
+| `test_actual_public_merged_omissions_and_all_chunk_statuses` | 3 / `gw3` | 1006.478 s | `native attempt deadline elapsed; retain all incomplete charge reservations` |
+
+For both cases, the JUnit failure's embedded result equals the archived `actual-result.json`, the worker's setup/call/teardown durations match the rounded JUnit duration, and all five saved capture pins match the actual archived bytes. The archived inventories include the root directory and its observed mode. Before, archived, and after inventories agree. These facts mean the snapshot was retained consistently; case completeness remains `not_assessed`.
+
+The neutral live summary leaves tool 4 pending. Its final transcript has a successful result for tool 4 followed by the next tool 5 callback request; control decisions stop at tool 4. The saved failure does not retain the stream role or earlier/later metadata pair that triggered the refusal. Final capture can include later bytes, and it does not establish a rewrite or race. The existing reader's equal-size/changed-mtime assumption requires separate investigation; Linux permits fields in one stat result to reflect different moments ([stat(2)](https://man7.org/linux/man-pages/man2/stat.2.html)).
+
+The omission case records sealed phase 1 and core, pending tool 19 (`native_shared_stage advance`), and no completed assembly or native terminal. Its JUnit time includes fixture setup and cleanup and is not an attempt-only measurement. Neither case supplies scientific, billing, provider, or full performance acceptance.
+
+The completed run failed. Shards 1 and 3 failed on the cases above; shards 2, 4, 5 and 6 passed. The six shard JUnit files contain 15,091 cases: 15,054 passed, 2 failed, 35 skipped and no errors. These totals exclude the separate offline evaluation, audit and canary jobs. `summary.json` records exact per-shard counts, artifact and XML pins, and controller outcomes. All six retained ZIPs were rechecked using the packaged verifier, reproducing their saved verification reports byte-for-byte. No artifact's absence from one worker implies that the node was globally unselected or unrun.
+
+## Recheck without native code or network access
+
+From the repository root, select a **new** output directory:
+
+```bash
+NATIVE_EVIDENCE_OUT=$(mktemp -d)
+NATIVE_EVIDENCE_NOTES=notes/native_ci_recovery_2026-10-08
+NATIVE_EVIDENCE_RUN="$NATIVE_EVIDENCE_NOTES/evidence/run-37759818157"
+for shard in 1 2 3 4 5 6; do
+  python3 -I -B -S "$NATIVE_EVIDENCE_NOTES/verify_native_ci_artifact.py" \
+    --archive "$NATIVE_EVIDENCE_RUN/shard${shard}.zip" \
+    --metadata "$NATIVE_EVIDENCE_RUN/shard${shard}-metadata.json" \
+    --expected "$NATIVE_EVIDENCE_RUN/expected.json" \
+    --output "$NATIVE_EVIDENCE_OUT/shard${shard}-verified.json"
+done
+```
+
+The standalone verifier uses only Python's standard library. It checks finite ZIP/tar membership and byte bounds, canonical member names, root/child modes and types, file digests, link targets, three inventories, exact source declarations, and fixed artifact availability. It follows no JSON authority paths, loads no archived Python, and performs no network or subprocess calls. Each output is created exclusively; an existing destination is refused. Saved application JSON is a separate diagnostic layer: malformed or partial results preserve an otherwise verified archive and produce `diagnostic_error` without inventing completion.
+
+Worker outcomes are retained by the verifier. JUnit-to-worker/result comparisons are separately recorded in the literal inspection summaries; the verifier itself does not certify pytest scheduling, JUnit completeness, or process execution authenticity.
+
+## Verifier regression controls
+
+Seven saved controls in `evidence/controls/controls.json` passed against the exact retained verifier and control-source hashes:
+
+- Stable archives with malformed JSON, truncated JSON, or a null live result retain their inventory checks and report a diagnostic error, while the second valid case is still inspected.
+- Valid local metadata at exactly 32 MiB is accepted; valid metadata at 32 MiB plus one byte is refused for size.
+- An overflowing numeric exponent (`1e999`) is refused; an ordinary finite duration is preserved.
+
+The three small synthetic ZIPs and their declarations/results are retained. Their source identities are explicitly invented and do not attest to repository or hosted execution. The two large size-bound JSON files are reproducible from the control script and are not committed.
+
+To rerun the controls outside the checkout, copy the two exact tools into another new directory:
+
+```bash
+NATIVE_CONTROL_OUT=$(mktemp -d)
+cp "$NATIVE_EVIDENCE_NOTES/verify_native_ci_artifact.py" \
+   "$NATIVE_EVIDENCE_NOTES/native_ci_verifier_controls.py" "$NATIVE_CONTROL_OUT/"
+python3 -I -B -S "$NATIVE_CONTROL_OUT/native_ci_verifier_controls.py"
+```
+
+The control script leaves a new directory containing all invented fixtures and the result ledger. It executes only the sibling verifier; it does not import repository modules or run the native acceptance cases. ZIP/gzip timestamps may make newly generated fixture hashes differ; each run records its own exact hashes.

notes/native_ci_recovery_2026-10-08/evidence/controls/controls.json

@@ -0,0 +1 @@
+{"control_source": {"bytes": 6673, "sha256": "33ef0b1d1ddbfe4c3891160af3c61a12395fcb5254adc1359f54fc0072ebb16b"}, "controls": [{"archive": {"bytes": 9251, "sha256": "1058a2e75afa1bbed3b71ee0f1a0d89fef4ce4f3848ff532636ffd5a7a1d4025"}, "name": "malformed", "passed": true}, {"archive": {"bytes": 9250, "sha256": "8bdcfb201f50a35383802b07904034273cc1a21a8629f0db89186904507d09ae"}, "name": "truncated", "passed": true}, {"archive": {"bytes": 9281, "sha256": "a6069c4c5f3b830216c3c7e7dcff5f9d09c28e2afe1648f0e27b281ccb088486"}, "name": "null_result", "passed": true}, {"name": "metadata_exact_32MiB_accepted", "passed": true}, {"name": "metadata_32MiB_plus_one_refused", "passed": true}, {"name": "exponent_overflow_refused", "passed": true}, {"name": "finite_duration_preserved", "passed": true}], "driver": {"bytes": 20608, "name": "verify_native_ci_artifact.py", "sha256": "2b18471d94520d1b6d954b9eb5936a46e507433f57651616c7cfc11f60c41f69"}, "format": "external_verifier_synthetic_controls_v1", "issue": 4679, "scope": "Invented metadata archives only. No native execution, repository imports, filesystem extraction or replay."}

notes/native_ci_recovery_2026-10-08/evidence/controls/malformed/expected.json

@@ -0,0 +1 @@
+{"files": {".github/workflows/main.yaml": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/fixtures/native_shared_execution/fake_cli.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/native_shared_fixture.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/native_shared_omission_fixture.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/test_native_shared_execution.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/test_native_shared_execution_variant.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "utils/native_ci_evidence.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "utils/native_shared_acceptance.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}}, "merge_sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "pr_head": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "run_id": 1, "tree_sha": "cccccccccccccccccccccccccccccccccccccccc"}

notes/native_ci_recovery_2026-10-08/evidence/controls/malformed/metadata.json

@@ -0,0 +1 @@
+{"digest": "sha256:1058a2e75afa1bbed3b71ee0f1a0d89fef4ce4f3848ff532636ffd5a7a1d4025", "id": 1, "name": "native-synthetic-cases-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-1-1-shard1", "size_in_bytes": 9251, "workflow_run": {"head_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "id": 1}}

notes/native_ci_recovery_2026-10-08/evidence/controls/malformed/verified.json

@@ -0,0 +1 @@
+{"archive": {"bytes": 9251, "sha256": "1058a2e75afa1bbed3b71ee0f1a0d89fef4ce4f3848ff532636ffd5a7a1d4025"}, "artifact_id": 1, "artifact_name": "native-synthetic-cases-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-1-1-shard1", "cases": [{"actual_archive": {"bytes": 243, "sha256": "dfc8254cd23dbd8aad41bd23a42c0ac113b71f732011e2d82788823a6e0eb965"}, "actual_inventory": {".": {"mode": 456, "type": "directory"}, "actual-result.json": {"bytes": 160, "mode": 416, "sha256": "f411a741f20adb722fe7c7cfbc6c1cb9a5f39a8ae44ce4a64aeb9bfc493087a0", "type": "file"}}, "archive_pin_matches": true, "availability": {"actual-result.json": "file", "authority/composition-execution.json": "absent", "authority/runtime-execution.json": "absent", "authority/runtime.json": "absent", "authority/selection.json": "absent", "fresh-attempt/stages/journal.json": "absent", "native-attempt/control.jsonl": "absent", "native-attempt/registration.json": "absent", "native-attempt/started.json": "absent", "native-attempt/stderr.txt": "absent", "native-attempt/transcript.jsonl": "absent"}, "before_archived_after_equal": true, "capture_status": "retained", "case_completeness": "not_assessed", "diagnostic": {"child_exit_code": null, "first_stop": null, "native_completion": "not_inferred", "phase_history": {"pending_tool_ids": []}, "result_available": true, "result_pin": {"bytes": 160, "sha256": "f411a741f20adb722fe7c7cfbc6c1cb9a5f39a8ae44ce4a64aeb9bfc493087a0"}, "runtime_gates_passed": false, "saved_capture_bindings": [], "scope": "Saved result and literal transcript inspection only; no replay, timing attribution or authenticated-execution claim.", "shutdown": null, "state": "failed", "transcript_available": false}, "manifest_pin": {"bytes": 3122, "sha256": "055d6f21cebdc1c00e78a0c47e60ac360251627aed80008204ec9241f5bdb961"}, "matches_archived_inventory": true, "node_id": "tests/test_native_shared_execution_variant.py::test_actual_public_merged_omissions_and_all_chunk_statuses", "test_outcome": {"call": {"duration_seconds": 0.25, "outcome": "failed"}, "teardown": {"duration_seconds": 0.01, "outcome": "passed"}}, "worker": "gw0"}, {"actual_archive": {"bytes": 142, "sha256": "fecc6504da7a5b6d1b5bc52be8303570b274d5af0153f9354e85c16c8a931163"}, "actual_inventory": {".": {"mode": 456, "type": "directory"}, "actual-result.json": {"bytes": 8, "mode": 416, "sha256": "62b8125a6f6d924ec53345b5fcd58ca3ed3f5e7d51e2e146e5f1346508acce69", "type": "file"}}, "archive_pin_matches": true, "availability": {"actual-result.json": "file", "authority/composition-execution.json": "absent", "authority/runtime-execution.json": "absent", "authority/runtime.json": "absent", "authority/selection.json": "absent", "fresh-attempt/stages/journal.json": "absent", "native-attempt/control.jsonl": "absent", "native-attempt/registration.json": "absent", "native-attempt/started.json": "absent", "native-attempt/stderr.txt": "absent", "native-attempt/transcript.jsonl": "absent"}, "before_archived_after_equal": true, "capture_status": "retained", "case_completeness": "not_assessed", "diagnostic": {"error": "JSONDecodeError: Expecting value: line 1 column 1 (char 0)", "native_completion": "not_inferred", "status": "diagnostic_error"}, "manifest_pin": {"bytes": 3104, "sha256": "db3d2d3b7c4d052357a161ada54607335ca2a53c2d1d7edfef69b7450226e08c"}, "matches_archived_inventory": true, "node_id": "tests/test_native_shared_execution.py::test_actual_public_correction_three_workers_and_saved_completion", "test_outcome": {"call": {"duration_seconds": 0.25, "outcome": "failed"}, "teardown": {"duration_seconds": 0.01, "outcome": "passed"}}, "worker": "gw0"}], "nodes_with_manifests": ["tests/test_native_shared_execution.py::test_actual_public_correction_three_workers_and_saved_completion", "tests/test_native_shared_execution_variant.py::test_actual_public_merged_omissions_and_all_chunk_statuses"], "nodes_without_manifests_in_this_shard": [], "pr_head": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "run_id": 1, "source": {"files_checked": 8, "head": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "tree": "cccccccccccccccccccccccccccccccccccccccc"}, "verification_scope": "Content consistency and declared GitHub/source bindings; not authenticated execution.", "worker_outcomes": {}}

notes/native_ci_recovery_2026-10-08/evidence/controls/null_result/expected.json

@@ -0,0 +1 @@
+{"files": {".github/workflows/main.yaml": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/fixtures/native_shared_execution/fake_cli.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/native_shared_fixture.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/native_shared_omission_fixture.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/test_native_shared_execution.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "tests/test_native_shared_execution_variant.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "utils/native_ci_evidence.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}, "utils/native_shared_acceptance.py": {"bytes": 27, "sha256": "18cc44a3121dd00f98a4e008db205404f2431680b45a86f098bf5d5448c8c21c"}}, "merge_sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "pr_head": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "run_id": 1, "tree_sha": "cccccccccccccccccccccccccccccccccccccccc"}

notes/native_ci_recovery_2026-10-08/evidence/controls/null_result/metadata.json

@@ -0,0 +1 @@
+{"digest": "sha256:a6069c4c5f3b830216c3c7e7dcff5f9d09c28e2afe1648f0e27b281ccb088486", "id": 1, "name": "native-synthetic-cases-bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-1-1-shard1", "size_in_bytes": 9281, "workflow_run": {"head_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "id": 1}}