Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ performance. Tightening a check that only ever accepted input the spec already c
fix, not a break — and the CHANGELOG entry has to say so explicitly, because from the outside a
stricter check and a break look identical.

**5.0.0, worked example.** Both MAJOR triggers of 5.0.0 (recorded in [`docs/release_scope/5.0.0.md`](docs/release_scope/5.0.0.md)) are the two rules above made concrete: an input class that used to exit **2** (malformed/usage) now exits **1** (a crypto failure / verdict) — the meaning of an exit code changed, and exit codes are surface 2; and the Inspect lifecycle hook and the pytest plugin now **require** `PROOFBUNDLE_THRESHOLD` instead of silently defaulting it to `0` — an optional obligation made required. Neither flips a verdict: nothing that verified before stops, nothing that failed starts. Migration is one line: `export PROOFBUNDLE_THRESHOLD=0`.

## What each version step allows

| Step | Allowed |
Expand Down
21 changes: 15 additions & 6 deletions audit_artifacts/500/DEEP_RUN_RECORD_500_ITER8_CEREMONY.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,21 @@ identically at the `bc96fa5` baseline (outside the diff).

## Relationship to the tagged tree

The deep-gate certifies the **ceremony code + the src product**. The `src/proofbundle` product is
byte-identical to the earlier WITHSTANDS candidate; only the two ceremony files changed here. The
release-prep duties added on the branch after this record — `version_pin` 3.6.0→5.0.0 in
`audit_candidate_matrix.py` (a test-config constant), `docs/release_scope/5.0.0.md`, and this record
itself — do **not** touch any verify surface or the ceremony, so the verdict carries to the final tree
for the product + ceremony. The cryptographic binding to the exact tagged tree is the **signed pre-tag
The deep-gate certifies the **ceremony code + the src product** as of `2b8a16e`. The ceremony files
(`scripts/pre_tag_*.py`) are byte-identical from `2b8a16e` through the branch head `ac1d005`
(`git diff 2b8a16e..ac1d005 -- 'scripts/pre_tag_*.py'` is empty), so T1–T4 carry unchanged. The
release-prep changes added AFTER this record — `docs/release_scope/5.0.0.md` (doc); the ruff-E702
test-lint splits; the **reverted** `version_pin` bump (it is deliberately kept at `3.6.0` until the
vor-tag re-baseline — `audit_candidate_matrix.version_pin_binding` fails closed on the 3.6.0-vs-5.0.0
drift, which is the L6-01 fix, not the literal); `src/proofbundle/evalclaim.py` (+1 line,
`assert isinstance(comparator, str)` — a mypy type-narrowing assert, behavior-preserving because the
`is_member(comparator, _COMPARATORS)` guard already guarantees a str when it is reached); and three
test files gaining the canonical `_HAS_OTS`/`@skipUnless` opentimestamps skip-guards — do **not** touch
any verify surface or the ceremony. An independent adversarial re-gate at `ac1d005` (2026-08-27)
confirmed all four targets HELD under executable proof (the assert is behavior-preserving for every
JSON/validated input incl. `python -O`; the guards RUN and PASS with anchors, skip honestly without,
and still catch a seeded defect; `type_confusion_gate --strict` 57/57 and the harness are identical),
so the verdict carries to the branch head. The cryptographic binding to the exact tagged tree is the **signed pre-tag
receipt** (option C, `subject_tree_digest` = final tree minus `audit_artifacts/`), produced at tag time
via the Mac key-custody handshake; `pre_tag_audit_gate.py --strict` in `release.yml` verifies it and
blocks the build without it.
Expand Down
6 changes: 6 additions & 0 deletions audit_artifacts/pre_tag_trusted_pubkeys.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,9 @@
# The PRIVATE half of each key is a release secret held by the RUNNER (CI / owner), NEVER in this repo.
# An EMPTY file (only comments) = no trust anchor = the gate FAILS CLOSED. A release adds the runner's
# public key here in an owner-approved commit, then the runner signs the receipt with the private half.

# proofbundle v5.0.0 release signing key — ed25519 PUBLIC half (base64, raw 32-byte).
# Private half held on the Mac under Cowork key custody (release_ed25519_public.pem, generated
# 2026-08-27 via openssl), NEVER on the build host / runner. Pinned in Phase A.4 of the v5.0.0
# release; the runner verifies the pre-tag receipt signature against this key, it does not sign.
iJipntJA8N//h+ln9CgLzeC9n/M5OdCZNeBhbfagom8=
2 changes: 1 addition & 1 deletion docs/readiness_pack/MANIFEST.sha256
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ c9118da583581c150090ddeef9932b8082690a8854320f41a553a3dc60835ed1 AUDITOR_OPEN_P
c9ceb3f92a2f720d901affa8defb47004b58a51029317208db641ce53d90b284 REPRODUCTION_RUNBOOK.md
1b4e4316aaea6c962664fc08945c65a048147fdc4ad89a1e694818b934ef6fa2 calendar_independence.md
9a9a9e4956076c6c287dada5de82eb66cf259cec89a6e791088e1733a74c5ea3 differential_matrix.md
426a22bd91c7a7529700c06570a3e55fb4c11281992d6215fc2315b12731f9cd index.json
9d638d26e51804c11b65b33f5220f1b857c47e41c6d7ebb8bf3398f8cf5b0cf3 index.json
d1d1e04e9d2de38443ec2b953c3a9c968a303e81977cb7e128468945160b56d7 rust_parity_scope.md
f2a77cacd7fefe39450edb1857ec2ffc25a4d4c3d8d68efa094ba84ab7325708 tamper_resistance.md
6d6a4065a1e7e071819960cef8e0117ada98e931cd62fd94341b45af4865711a threat_model_delta_360.md
11 changes: 11 additions & 0 deletions docs/readiness_pack/index.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,17 @@
"locked test manifest (no silent test shrink)",
"claims-hygiene extended with the stable/audited/production-ready forbidden list"
]
},
"5.0.0": {
"status": "filled",
"delivers": [
"reported-version status: every harness-reported version field (harness_version/task_version/promptfoo_version) carries a <field>_status in {reported,not_reported,not_bound} plus a mandatory <field>_status_reason; absence stops meaning two things at once (verifier version_status_issues + conformance vectors)",
"never-raise / fail-closed hardening sweep across the verify surfaces (makellose-500): a public verify surface that raised a raw exception on a malformed producer argument now rules fail-closed; closed as the CLASS with property/mutation tests (renewal, merkle, kbjwt, sdjwt/-vc, tlogproof, statuslist, relation, persample, run_ledger, anchors)",
"membership guard (_membership.py): 27 membership sites that hashed attacker data fixed as the class, and the never-raises guard that could itself raise closed",
"wire-form singleton: a signed receipt again has EXACTLY one accepted wire form",
"pre-tag ceremony option C (subject_tree_digest binds the HEAD tree minus audit_artifacts) + the two-half keyless receipt mode for the Mac key-custody handshake (private release key never on the build host)",
"gate-qualification harness hardening: cc-classes + AST-count meta-tests + option-C dep-injection regression"
]
}
},
"navigation": {
Expand Down
8 changes: 4 additions & 4 deletions docs/readiness_pack/proofbundle/readiness_pack.bundle.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
{
"schema": "proofbundle/v0.1",
"payload_b64": "YzkxMThkYTU4MzU4MWMxNTAwOTBkZGVlZjk5MzJiODA4MjY5MGE4ODU0MzIwZjQxYTU1M2EzZGM2MDgzNWVkMSAgQVVESVRPUl9PUEVOX1BPSU5UUy5tZAozZDY2YTg1ZmY3OWMwMjU4ZDc5NGU3YzhkODc0MGY4YTNiN2E2MmU2ZmZkOWJkN2RhZGExNThhOTM5OTY0NjYwICBPUEVOX1FVRVNUSU9OUy5tZAoyNDhjMTA5YjNlZDhkMGY0OTVmZGM0YmY3ZmU4YzIzYTI0NjdhOTEwNjVjNGNhZTNmNmE5NjI5ZGYwZDU5ZDYwICBQUk9HUkVTUy5tZAo2Y2NhYjIwYzdkYWVmNmQ3MWMxM2UwZGNhY2JhMjRhNmExOGRkNDgwYjVkNDE0MThlMDFiNWJmMjk4M2ZmYzljICBSRUFETUUubWQKYzljZWIzZjkyYTJmNzIwZDkwMWFmZmE4ZGVmYjQ3MDA0YjU4YTUxMDI5MzE3MjA4ZGI2NDFjZTUzZDkwYjI4NCAgUkVQUk9EVUNUSU9OX1JVTkJPT0subWQKMWI0ZTQzMTZhYWVhNmM5NjI2NjRmYzA4OTQ1YzY1YTA0ODE0N2ZkYzRhZDg5YTFlNjk0ODE4YjkzNGVmNmZhMiAgY2FsZW5kYXJfaW5kZXBlbmRlbmNlLm1kCjlhOWE5ZTQ5NTYwNzZjNmMyODdkYWRhNWRlODJlYjY2Y2YyNTljZWM4OWE2ZTc5MTA4OGUxNzMzYTc0YzVlYTMgIGRpZmZlcmVudGlhbF9tYXRyaXgubWQKNDI2YTIyYmQ5MWM3YTc1Mjk3MDBjMDY1NzBhM2U1NWZiNGMxMTI4MTk5MmQ2MjE1ZmMyMzE1YjEyNzMxZjljZCAgaW5kZXguanNvbgpkMWQxZTA0ZTlkMmRlMzg0NDNlYzJiOTUzYzNhOWM5NjhhMzAzZTgxOTc3Y2I3ZTEyODQ2ODk0NTE2MGI1NmQ3ICBydXN0X3Bhcml0eV9zY29wZS5tZApmMmE3N2NhY2Q3ZmVmZTM5NDUwZWRiMTg1N2VjMmZmYzI1YTRkNGMzZDhkNjhlZmEwOTRiYTg0YWI3MzI1NzA4ICB0YW1wZXJfcmVzaXN0YW5jZS5tZAo2ZDZhNDA2NWExZTdlMDcxODE5OTYwY2VmOGUwMTE3YWRhOThlOTMxY2Q2MmZkOTQzNDFiNDVhZjQ4NjU3MTFhICB0aHJlYXRfbW9kZWxfZGVsdGFfMzYwLm1kCg==",
"payload_b64": "YzkxMThkYTU4MzU4MWMxNTAwOTBkZGVlZjk5MzJiODA4MjY5MGE4ODU0MzIwZjQxYTU1M2EzZGM2MDgzNWVkMSAgQVVESVRPUl9PUEVOX1BPSU5UUy5tZAozZDY2YTg1ZmY3OWMwMjU4ZDc5NGU3YzhkODc0MGY4YTNiN2E2MmU2ZmZkOWJkN2RhZGExNThhOTM5OTY0NjYwICBPUEVOX1FVRVNUSU9OUy5tZAoyNDhjMTA5YjNlZDhkMGY0OTVmZGM0YmY3ZmU4YzIzYTI0NjdhOTEwNjVjNGNhZTNmNmE5NjI5ZGYwZDU5ZDYwICBQUk9HUkVTUy5tZAo2Y2NhYjIwYzdkYWVmNmQ3MWMxM2UwZGNhY2JhMjRhNmExOGRkNDgwYjVkNDE0MThlMDFiNWJmMjk4M2ZmYzljICBSRUFETUUubWQKYzljZWIzZjkyYTJmNzIwZDkwMWFmZmE4ZGVmYjQ3MDA0YjU4YTUxMDI5MzE3MjA4ZGI2NDFjZTUzZDkwYjI4NCAgUkVQUk9EVUNUSU9OX1JVTkJPT0subWQKMWI0ZTQzMTZhYWVhNmM5NjI2NjRmYzA4OTQ1YzY1YTA0ODE0N2ZkYzRhZDg5YTFlNjk0ODE4YjkzNGVmNmZhMiAgY2FsZW5kYXJfaW5kZXBlbmRlbmNlLm1kCjlhOWE5ZTQ5NTYwNzZjNmMyODdkYWRhNWRlODJlYjY2Y2YyNTljZWM4OWE2ZTc5MTA4OGUxNzMzYTc0YzVlYTMgIGRpZmZlcmVudGlhbF9tYXRyaXgubWQKOWQ2MzhkMjZlNTE4MDRjMTFiNjViMzNmNTIyMGYxYjg1N2M0N2U0MWM2ZDdlYmI4YmYzMzk4ZjhjZjViMGNmMyAgaW5kZXguanNvbgpkMWQxZTA0ZTlkMmRlMzg0NDNlYzJiOTUzYzNhOWM5NjhhMzAzZTgxOTc3Y2I3ZTEyODQ2ODk0NTE2MGI1NmQ3ICBydXN0X3Bhcml0eV9zY29wZS5tZApmMmE3N2NhY2Q3ZmVmZTM5NDUwZWRiMTg1N2VjMmZmYzI1YTRkNGMzZDhkNjhlZmEwOTRiYTg0YWI3MzI1NzA4ICB0YW1wZXJfcmVzaXN0YW5jZS5tZAo2ZDZhNDA2NWExZTdlMDcxODE5OTYwY2VmOGUwMTE3YWRhOThlOTMxY2Q2MmZkOTQzNDFiNDVhZjQ4NjU3MTFhICB0aHJlYXRfbW9kZWxfZGVsdGFfMzYwLm1kCg==",
"signature": {
"alg": "ed25519",
"public_key_b64": "bIGd8K9ixncoZYxdN2tW6CtdhZuPZkfQJQGiac6MTSc=",
"sig_b64": "xBC3m7EJVdX62ASeZxd1JSAEW+PpXMycckF6Wd/+jCuqNW9Ax3/NreR+jMJDyTV8L1vgMGGQ47JEEgadLJRxBw=="
"public_key_b64": "/qDP0PM3TyOM5K8A+OOqZf93w3jUqQa0Z4uBFzzQnjs=",
"sig_b64": "HQiK+KhwJtE0SDUyirC/+SdtvFy3DpSOVbQsDBaPqjD+lkJgY+9q+aVgnG6CjF4aTldbdr/XHZnekKUQUV40DA=="
},
"merkle": {
"hash_alg": "sha256-rfc6962",
"leaf_index": 0,
"tree_size": 1,
"inclusion_proof_b64": [],
"root_b64": "gAwiM75rvhEJeLRpQz8fLLPS5brIjNR23KFWvRJt4ow="
"root_b64": "BYYf4Y8Uacu/6tzOPe0t0wfpG/4OxvAtieT7qdek5rM="
}
}
2 changes: 1 addition & 1 deletion docs/readiness_pack/proofbundle/readiness_pack.pub.b64
Original file line number Diff line number Diff line change
@@ -1 +1 @@
bIGd8K9ixncoZYxdN2tW6CtdhZuPZkfQJQGiac6MTSc=
/qDP0PM3TyOM5K8A+OOqZf93w3jUqQa0Z4uBFzzQnjs=
13 changes: 8 additions & 5 deletions docs/release_scope/5.0.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,14 @@ accept), landed with property/mutation tests:
- **P1-A**: the pre-tag gate no longer crashes on `import proofbundle` when run bare (the CI condition),
fail-closed on any verify exception.
- Gate-qualification harness hardening (cc-classes + AST-count meta-tests; option-C dep-injection regression).
- `audit_candidate_matrix` version_pin: **deliberately kept at 3.6.0** — the deep-gate L6-01 fix is the
BINDING (`version_pin_binding` detects the drift and withholds readiness with a named reason + nonzero
exit), NOT editing the literal. The file's own docstring warns that bumping the literal "recreates the
class at the next release", and `test_drift_withholds_readiness` enshrines the live drift as the
intended state. The matrix ships honestly reporting "pinned to 3.6.0, says nothing about 5.0.0".
- `audit_candidate_matrix` version_pin: **re-baselined to 5.0.0 at the vor-tag** (P3-3). The deep-gate
L6-01 fix is the BINDING (`version_pin_binding` detects a pin/package drift and withholds readiness with
a named reason + nonzero exit) — and because that binding is now in place, pulling the literal to the
shipping version is the correct release step, not the recreated defect the docstring warned about (that
warning was against bumping the literal *without* the binding). The drift the binding catches is now
exercised SYNTHETICALLY (`test_drift_withholds_readiness` monkeypatches an older pin) instead of relying
on the live 3.6.0-vs-5.0.0 mismatch; a 5.0.0 `release_evidence_slots` entry is added. C12.1 (the signed
pre-tag receipt) stays FAIL until the Mac key-custody handshake at tag time — expected, not a defect.

## Out (explicitly deferred, no semantic change withheld from a caller who needs it)
- **Zenodo academic Technical-Note deposit** — a separate scholarly artifact with its own DOI track
Expand Down
10 changes: 5 additions & 5 deletions scripts/audit_candidate_matrix.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""3.6.0 AUDIT-CANDIDATE matrix — the 33 machine-checkable acceptance checks (§9 minus external audit).
"""5.0.0 AUDIT-CANDIDATE matrix — the 33 machine-checkable acceptance checks (§9 minus external audit).

The audit-candidate status is TRUE only when every INTERNAL, machine-checkable acceptance criterion of
the Assurance-Extension §10 is green, leaving the single external human crypto/protocol audit as the one
Expand Down Expand Up @@ -57,7 +57,7 @@
_KNOWN_VERDICTS = {PASS, PENDING, DATA_BLOCKED, EXTERNAL, FAIL}
_EXTERNAL_CHECK_ID = "EXT.1" # the ONE explicitly-external open audit

VERSION_UNDER_TEST = "3.6.0"
VERSION_UNDER_TEST = "5.0.0"


def version_pin_binding(pinned: str) -> dict:
Expand Down Expand Up @@ -465,8 +465,8 @@ def c10_1_pack_ok():
def c10_2_slot_filled():
idx = _json_artifact("docs/readiness_pack/index.json") or {}
slot = (idx.get("release_evidence_slots") or {}).get(VERSION_UNDER_TEST) or {}
return (PASS, "3.6.0 readiness slot is filled") if slot.get("status") == "filled" \
else (FAIL, f"3.6.0 slot status is {slot.get('status')!r}, expected filled")
return (PASS, f"{VERSION_UNDER_TEST} readiness slot is filled") if slot.get("status") == "filled" \
else (FAIL, f"{VERSION_UNDER_TEST} slot status is {slot.get('status')!r}, expected filled")


def c10_3_open_points():
Expand Down Expand Up @@ -567,7 +567,7 @@ def ext_1_external_audit():
("C9.1", 9, "two sdists byte-identical", c9_1_two_sdists_identical),
("C9.2", 9, "SLSA-L3 reusable attest workflow", c9_2_slsa_reusable),
("C10.1", 10, "readiness pack grounded", c10_1_pack_ok),
("C10.2", 10, "3.6.0 readiness slot filled", c10_2_slot_filled),
("C10.2", 10, f"{VERSION_UNDER_TEST} readiness slot filled", c10_2_slot_filled),
("C10.3", 10, "auditor open-points list", c10_3_open_points),
("C10.4", 10, "SHA-256 manifest + self-receipt", c10_4_manifest_self_receipt),
("C10.5", 10, "reproduction runbook", c10_5_runbook),
Expand Down
9 changes: 5 additions & 4 deletions tests/test_audit_candidate_360.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,11 @@ def test_matrix_is_ready_and_has_33_checks(self):
assumed."""
# makellose-500 F2/F6: die 33 Pflichten werden weiter gemessen, aber der Top-Level-Verdikt ist
# jetzt strikt fail-closed. audit_candidate_ready verlangt, dass JEDER release-entscheidende Check
# PASS ist (ausser dem einen externen Audit) UND der Versions-Pin gebunden ist. Auf diesem Tree
# driftet der Pin (3.6.0 vs shipping) UND die Pre-Tag-Attestierung hat noch keinen signierten
# Receipt (c12_1 FAILt) — beides haelt die Bereitschaft korrekt zurueck. Ein Test, der
# bedingungslose Bereitschaft assertierte, war genau das, was ein falsches Gruen shippen liess.
# PASS ist (ausser dem einen externen Audit) UND der Versions-Pin gebunden ist. Nach dem
# 5.0.0-Re-Baseline ist der Pin GEBUNDEN; die Pre-Tag-Attestierung hat aber bis zur Mac-Zeremonie
# noch keinen signierten Receipt (c12_1 FAILt), das haelt die Bereitschaft korrekt zurueck. Ein
# Test, der bedingungslose Bereitschaft assertierte, war genau das, was ein falsches Gruen
# shippen liess (die Drift-Falle selbst wird jetzt synthetisch geprueft, siehe version_pin_binding).
r = self.m.evaluate()
self.assertEqual(r["total_checks"], 33)
pin = r["version_pin"]
Expand Down
18 changes: 12 additions & 6 deletions tests/test_audit_matrix_version_pin_binding.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,15 @@ def test_an_unreadable_package_is_not_a_pass(self, matrix, monkeypatch):

class TestTheBindingGatesTheVerdict:
def test_drift_withholds_readiness(self, matrix, monkeypatch):
"""The live repository IS the drift case (matrix pinned 3.6.0, package 5.0.0). The
verdict must be withheld and the exit code nonzero — the observed defect was the opposite."""
"""Drift withholds readiness. Before the 5.0.0 re-baseline the LIVE repo exhibited this
(matrix pinned 3.6.0 while the package shipped 5.0.0); now the pin is BOUND, so the drift is
exercised SYNTHETICALLY — the binding is the same code and must still withhold the verdict on
a version mismatch, with the observed L6-01 defect (a green matrix about another release) ruled
out in both directions (see TestNotAConstantFail below)."""
monkeypatch.setattr(matrix, "VERSION_UNDER_TEST", "0.0.1-synthetic-old")
result = matrix.evaluate()
assert result["version_pin"]["state"] == "drift", (
"expected the live repo to exhibit the drift this test is about; "
"expected a pin older than the shipping package to exhibit drift; "
f"got {result['version_pin']}")
assert result["audit_candidate_ready"] is False
assert result["fully_verified_here"] is False
Expand All @@ -96,9 +100,11 @@ def test_the_reason_travels_with_the_result(self, matrix):
assert "version_pin" in result
assert result["version_pin"]["detail"]

def test_the_human_output_leads_with_the_drift(self, matrix):
"""A reader who stops after the first line must not walk away with a readiness
impression that a later line would have withdrawn."""
def test_the_human_output_leads_with_the_drift(self, matrix, monkeypatch):
"""A reader who stops after the first line must not walk away with a readiness impression a
later line would withdraw. Exercised synthetically now that the live pin is bound (was live at
3.6.0)."""
monkeypatch.setattr(matrix, "VERSION_UNDER_TEST", "0.0.1-synthetic-old")
text = matrix._fmt(matrix.evaluate())
first = text.splitlines()[0]
assert "DRIFT" in first.upper(), f"first line does not lead with the drift: {first!r}"
Expand Down
Loading