diff --git a/.claude/skills/apr-dogfood/SKILL.md b/.claude/skills/apr-dogfood/SKILL.md index 226a3bb07e..3221b34fde 100644 --- a/.claude/skills/apr-dogfood/SKILL.md +++ b/.claude/skills/apr-dogfood/SKILL.md @@ -1591,7 +1591,7 @@ The nine clusters at zero, largest first: `http-orchestrate-banco` (95), `test-harness` (49), `rag-eval` (44), `qa-cgp` (37), `simulation` (18), `orchestrate-pacha-secrets` (17). -**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 838 features gated (17.2%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*. +**Report both numbers or neither (T2).** "5 of 14 clusters gated (35.7%)" without "144 of 864 features gated (16.7%)" beside it is a proxy masquerading as coverage, and the gate refuses to emit it — *on this line too*. The rule is about the NUMBER, not about a phrasing, and it is enforced on every surface that can emit one: the gate's own report, the receipt, the output of diff --git a/.claude/skills/pr-review/SKILL.md b/.claude/skills/pr-review/SKILL.md index fd15cd5abe..01dcf7679b 100644 --- a/.claude/skills/pr-review/SKILL.md +++ b/.claude/skills/pr-review/SKILL.md @@ -22,7 +22,7 @@ difference is a defect in this file. **Contract**: `contracts/pr-review-skill-v2.yaml` (§1 grounding, §7 blocking, §8 metrics) **Guard**: `scripts/check_pr_review_receipt.sh` — it validates what you emit here, it has its own positive controls, and its mutation set (`scripts/mutate-guard.sh`) reports -233/233. **Run it on your own receipt before you post anything.** +237/237. **Run it on your own receipt before you post anything.** ## Context @@ -438,7 +438,7 @@ Bash guards are exercised with `bats-core` fixtures. For the receipt guard itsel mutation set already exists and is a derivation, not a list: ```bash -bash scripts/mutate-guard.sh # 233/233 on scripts/check_pr_review_receipt.sh +bash scripts/mutate-guard.sh # 237/237 on scripts/check_pr_review_receipt.sh ``` **`attempted: 0` with `status: consulted` is rejected** (fixture row 2). A mutation set @@ -1192,7 +1192,7 @@ the real merged commits, not from the spec's reasoning about them. ## §13 Autonomous merge on quorum (DESIGNED AND BUILT, **NOT ARMED**) Spec §13. Operator instruction, 2026-08-31: PRs auto-merge once the review quorum passes. -The mechanism exists — `scripts/pr_review_quorum_arm.sh`, a table of 83 rows, a 134-mutant +The mechanism exists — `scripts/pr_review_quorum_arm.sh`, a table of 90 rows, a 134-mutant set at 100% — and **it is reachable from no workflow that can merge anything.** §13.11 is the arming ladder; rung 0 is where this file is written. @@ -1345,7 +1345,7 @@ That is the first falsifiable property of the section, and `q-44` plus the ### §13.9 Verifying the mechanism ```bash -bats tests/pr-review-quorum.bats # 83 rows: one per refusal path, four that PERMIT +bats tests/pr-review-quorum.bats # 90 rows: one per refusal path, four that PERMIT bash scripts/mutate_quorum_arm.sh # 134/134 — §13.10 fixes this at one, no ratchet bash scripts/mutate_quorum_arm.sh --list # the catalogue, no mutants run ``` diff --git a/.github/actionlint.yaml b/.github/actionlint.yaml index f1162eaef4..074889486a 100644 --- a/.github/actionlint.yaml +++ b/.github/actionlint.yaml @@ -24,3 +24,5 @@ self-hosted-runner: - apple-silicon - m4 - mini + # #4292 / infra#1042: the mini's dedicated repo-level release runner + - mini-builder diff --git a/.github/workflows/binary-release.yml b/.github/workflows/binary-release.yml index 7bfcbc9e91..ea7e8c357c 100644 --- a/.github/workflows/binary-release.yml +++ b/.github/workflows/binary-release.yml @@ -79,6 +79,34 @@ concurrency: jobs: + # A release that already carries every asset is NOT rebuilt (#4286). A promoted final + # is its rc's exact bytes (scripts/release/promote_rc.sh); publishing it with a + # personal token fires `release: published` here, and the upload steps below delete + # and replace same-named assets, so an ungated rebuild would silently swap the bytes + # that were dogfooded for new ones. With all eighteen present the build jobs skip and + # verify + smoke still run, on the promoted bytes. A partial release (a lane died) + # still rebuilds. An unreadable release (exit 2) fails this job, so the builds skip: + # when in doubt, never overwrite. + assets: + name: Does the release already carry every asset? + runs-on: [self-hosted, Linux, X64, clean-room] + timeout-minutes: 5 + outputs: + present: ${{ steps.check.outputs.present }} + steps: + - uses: actions/checkout@v7 + - id: check + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + TAG: ${{ github.event.release.tag_name || inputs.tag }} + run: | + bash scripts/check_release_assets.sh "$TAG"; rc=$? + case "$rc" in + 0) echo "present=true" >> "$GITHUB_OUTPUT"; echo "$TAG already carries every asset: nothing is rebuilt" >> "$GITHUB_STEP_SUMMARY" ;; + 1) echo "present=false" >> "$GITHUB_OUTPUT" ;; + *) echo "::error::could not read release $TAG (rc=$rc); refusing to build over assets that may exist"; exit 1 ;; + esac + # The pv lane on the fleet (#3073; operator rule 2026-09-10: no GitHub-hosted runners). It ran on # ubuntu-latest, with cross for aarch64. Each target now builds natively on the box of its architecture # (x86_64 on yoga, aarch64 on gx10) in a sibling container, the pattern of build-apr-cpu: gnu targets @@ -86,6 +114,8 @@ jobs: # well, so no floor rises), musl targets inside rust:1.93.0-bookworm (static). Asset names unchanged. build: name: pv ${{ matrix.target }} on ${{ matrix.host }} + needs: assets + if: needs.assets.outputs.present == 'false' runs-on: ${{ fromJSON(matrix.labels) }} strategy: fail-fast: false @@ -124,6 +154,20 @@ jobs: with: ref: ${{ steps.tag.outputs.tag }} + # #4110/#4290: an rc asset's crate version IS its tag. The rc commit's Cargo.toml says + # X.Y.Z, so without this every vX.Y.Z-rc.N asset printed bare `X.Y.Z` (v0.69.3-rc.2: + # `apr 0.69.3 (v0.69.3+no-git)`). The stamper comes from THIS workflow's commit, not the + # tag's tree, which an rc cut from release/X.Y.Z predates. A final tag stamps nothing. + # verify-apr-assets then requires `apr --version` to equal the tag exactly. + - name: Stamp the rc version into the tag tree (#4110) + env: + TAG: ${{ steps.tag.outputs.tag }} + run: | + case "$TAG" in *-rc.*) ;; *) echo "final tag $TAG: nothing to stamp"; exit 0 ;; esac + git fetch --no-tags --depth 1 origin "$GITHUB_WORKFLOW_SHA" + git show "FETCH_HEAD:scripts/release/stamp_rc_version.sh" > "$RUNNER_TEMP/stamp_rc_version.sh" + bash "$RUNNER_TEMP/stamp_rc_version.sh" "$GITHUB_WORKSPACE" "$TAG" + - name: Preflight - this box has the tools this job assumes run: bash scripts/ci_self_hosted_preflight.sh --need docker objdump @@ -220,7 +264,10 @@ jobs: run: | TAG="${{ steps.tag.outputs.tag }}" API="https://api.github.com/repos/${GITHUB_REPOSITORY}" - rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases/tags/$TAG") + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327), + # and releases/tags/ never returns a draft + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") rid=$(printf '%s' "$rel" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])') for f in "${{ steps.package.outputs.archive }}" "${{ steps.package.outputs.sha }}"; do # --clobber semantics: delete an asset of the same name first @@ -233,6 +280,8 @@ jobs: build-apr-cuda: name: apr (cuda) ${{ matrix.target }} on ${{ matrix.host }} + needs: assets + if: needs.assets.outputs.present == 'false' runs-on: ${{ fromJSON(matrix.labels) }} strategy: fail-fast: false @@ -261,6 +310,20 @@ jobs: with: ref: ${{ steps.tag.outputs.tag }} + # #4110/#4290: an rc asset's crate version IS its tag. The rc commit's Cargo.toml says + # X.Y.Z, so without this every vX.Y.Z-rc.N asset printed bare `X.Y.Z` (v0.69.3-rc.2: + # `apr 0.69.3 (v0.69.3+no-git)`). The stamper comes from THIS workflow's commit, not the + # tag's tree, which an rc cut from release/X.Y.Z predates. A final tag stamps nothing. + # verify-apr-assets then requires `apr --version` to equal the tag exactly. + - name: Stamp the rc version into the tag tree (#4110) + env: + TAG: ${{ steps.tag.outputs.tag }} + run: | + case "$TAG" in *-rc.*) ;; *) echo "final tag $TAG: nothing to stamp"; exit 0 ;; esac + git fetch --no-tags --depth 1 origin "$GITHUB_WORKFLOW_SHA" + git show "FETCH_HEAD:scripts/release/stamp_rc_version.sh" > "$RUNNER_TEMP/stamp_rc_version.sh" + bash "$RUNNER_TEMP/stamp_rc_version.sh" "$GITHUB_WORKSPACE" "$TAG" + # FIRST thing this box does after it has a working tree. Run 34448908554 # built a 21 MB CUDA asset here and then died on `gh: command not found` # at the upload — forty minutes of GPU spent on a shell error a one-second @@ -296,6 +359,7 @@ jobs: -w /workspace \ -e CARGO_TARGET_DIR=/workspace/target \ -e CARGO_INCREMENTAL=0 \ + -e APR_GIT_SHA_OVERRIDE="$(git rev-parse --short=9 HEAD)" \ rust:1.93.0-bullseye \ sh -c 'set -e; ldd --version | head -1 > /workspace/target/BUILDER_GLIBC; rustc --version > /workspace/target/BUILDER_RUSTC; cargo build --release -p apr-cli --bin apr --target ${{ matrix.target }} --features cuda --locked; strip target/${{ matrix.target }}/release/apr' # the mounted target dir is the host cache; expose the binary where the next steps expect it @@ -348,7 +412,10 @@ jobs: run: | TAG="${{ steps.tag.outputs.tag }}" API="https://api.github.com/repos/${GITHUB_REPOSITORY}" - rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases/tags/$TAG") + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327), + # and releases/tags/ never returns a draft + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") rid=$(printf '%s' "$rel" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])') for f in "${{ steps.package.outputs.archive }}" "${{ steps.package.outputs.sha }}"; do # --clobber semantics: delete an asset of the same name first @@ -379,6 +446,8 @@ jobs: # same step, so the merge moves both or neither. build-apr-cpu: name: apr (cpu) ${{ matrix.target }} on ${{ matrix.host }} + needs: assets + if: needs.assets.outputs.present == 'false' runs-on: ${{ fromJSON(matrix.labels) }} strategy: fail-fast: false @@ -407,6 +476,20 @@ jobs: with: ref: ${{ steps.tag.outputs.tag }} + # #4110/#4290: an rc asset's crate version IS its tag. The rc commit's Cargo.toml says + # X.Y.Z, so without this every vX.Y.Z-rc.N asset printed bare `X.Y.Z` (v0.69.3-rc.2: + # `apr 0.69.3 (v0.69.3+no-git)`). The stamper comes from THIS workflow's commit, not the + # tag's tree, which an rc cut from release/X.Y.Z predates. A final tag stamps nothing. + # verify-apr-assets then requires `apr --version` to equal the tag exactly. + - name: Stamp the rc version into the tag tree (#4110) + env: + TAG: ${{ steps.tag.outputs.tag }} + run: | + case "$TAG" in *-rc.*) ;; *) echo "final tag $TAG: nothing to stamp"; exit 0 ;; esac + git fetch --no-tags --depth 1 origin "$GITHUB_WORKFLOW_SHA" + git show "FETCH_HEAD:scripts/release/stamp_rc_version.sh" > "$RUNNER_TEMP/stamp_rc_version.sh" + bash "$RUNNER_TEMP/stamp_rc_version.sh" "$GITHUB_WORKSPACE" "$TAG" + # 67-C2: the first thing this box does after it has a working tree. No # `--cuda` here — a CPU asset must build on a box with no visible device, # and asserting one would make this lane lie about what it needs. `docker` @@ -441,6 +524,7 @@ jobs: -w /workspace \ -e CARGO_TARGET_DIR=/workspace/target \ -e CARGO_INCREMENTAL=0 \ + -e APR_GIT_SHA_OVERRIDE="$(git rev-parse --short=9 HEAD)" \ rust:1.93.0-bullseye \ sh -c 'set -e; ldd --version | head -1 > /workspace/target/BUILDER_GLIBC; rustc --version > /workspace/target/BUILDER_RUSTC; cargo build --release -p apr-cli --bin apr --target ${{ matrix.target }} --locked; strip target/${{ matrix.target }}/release/apr' # the mounted target dir is the host cache; expose the binary where the next steps expect it @@ -515,7 +599,10 @@ jobs: run: | TAG="${{ steps.tag.outputs.tag }}" API="https://api.github.com/repos/${GITHUB_REPOSITORY}" - rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases/tags/$TAG") + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327), + # and releases/tags/ never returns a draft + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") rid=$(printf '%s' "$rel" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])') for f in "${{ steps.package.outputs.archive }}" "${{ steps.package.outputs.sha }}"; do # --clobber semantics: delete an asset of the same name first @@ -526,6 +613,101 @@ jobs: | python3 -c 'import json,sys; a=json.load(sys.stdin); print("uploaded", a["name"], a["size"], "bytes")' done + # mini (Darwin arm64) is fleet hardware and could not take v0.69.3-rc.2: the release shipped + # no darwin asset, so mini was RED on every rc (#4327, infra-64 evidence/fleet-versions.md). + # Native build on the self-hosted mini-m4 runner (no hosted runners, #3073). No container: + # macOS has no glibc floor to pin, and the stamped version + the sha are checked below + # before anything is uploaded. + build-apr-darwin: + name: apr (cpu) aarch64-apple-darwin on mini + needs: assets + if: needs.assets.outputs.present == 'false' + # mini-builder (#4292, infra#1042): the mini's dedicated repo-level release runner, used + # only through this label so it never joins the general pool. + runs-on: [self-hosted, macOS, ARM64, mini-builder] + env: + # The macOS floor, recorded in the Mach-O load command and read back below (#4292). + MACOSX_DEPLOYMENT_TARGET: "11.0" + steps: + - name: Resolve release tag + id: tag + run: | + TAG="${{ github.event.release.tag_name || inputs.tag }}" + [ -n "$TAG" ] || { echo "::error::No tag resolved"; exit 1; } + echo "tag=$TAG" >> "$GITHUB_OUTPUT" + + - name: Checkout at tag + uses: actions/checkout@v7 + with: + ref: ${{ steps.tag.outputs.tag }} + + # The release scripts come from THIS workflow's commit: the tag tree predates them. + - name: Stamp the rc version into the tag tree (#4110) + env: + TAG: ${{ steps.tag.outputs.tag }} + run: | + git fetch --no-tags --depth 1 origin "$GITHUB_WORKFLOW_SHA" + for f in stamp_rc_version.sh asset_version_check.sh; do + git show "FETCH_HEAD:scripts/release/$f" > "$RUNNER_TEMP/$f" + done + case "$TAG" in *-rc.*) ;; *) echo "final tag $TAG: nothing to stamp"; exit 0 ;; esac + bash "$RUNNER_TEMP/stamp_rc_version.sh" "$GITHUB_WORKSPACE" "$TAG" + + - name: Build apr on the default features (no cuda) + run: | + APR_GIT_SHA_OVERRIDE="$(git rev-parse --short=9 HEAD)" \ + cargo build --release -p apr-cli --bin apr --target aarch64-apple-darwin --locked + strip target/aarch64-apple-darwin/release/apr + + - name: The binary names its tag and commit (#4110) + run: | + line=$(target/aarch64-apple-darwin/release/apr --version) + bash "$RUNNER_TEMP/asset_version_check.sh" "${{ steps.tag.outputs.tag }}" "$(git rev-parse HEAD)" "$line" + + # The same discriminator as the linux cpu lane, plus the arch and the minimum + # macOS actually recorded in the binary (#4292). + - name: Prove the asset is an arm64 CPU build + run: | + BIN=target/aarch64-apple-darwin/release/apr + n=$(grep -ac 'libcuda.so' "$BIN" || true) + if [ "$n" -ne 0 ]; then + echo "::error::$BIN carries $n libcuda.so loader string(s) — this is a CUDA build wearing a -cpu name" + exit 1 + fi + file "$BIN" | grep -q 'arm64' || { echo "::error::$BIN is not arm64: $(file "$BIN")"; exit 1; } + MINOS=$(otool -l "$BIN" | awk '/LC_BUILD_VERSION/{f=1} f&&$1=="minos"{print $2; exit}') + echo "- minimum macOS (LC_BUILD_VERSION minos): ${MINOS:-unknown}" >> "$GITHUB_STEP_SUMMARY" + + - name: Package archive + id: package + run: | + ARCHIVE="apr-${{ steps.tag.outputs.tag }}-aarch64-apple-darwin-cpu" + mkdir -p "$ARCHIVE" + cp target/aarch64-apple-darwin/release/apr "$ARCHIVE/" + for f in README.md LICENSE LICENSE-MIT LICENSE-APACHE; do [ -f "$f" ] && cp "$f" "$ARCHIVE/"; done + tar czf "${ARCHIVE}.tar.gz" "$ARCHIVE" + shasum -a 256 "${ARCHIVE}.tar.gz" > "${ARCHIVE}.tar.gz.sha256" + echo "archive=${ARCHIVE}.tar.gz" >> "$GITHUB_OUTPUT" + echo "sha=${ARCHIVE}.tar.gz.sha256" >> "$GITHUB_OUTPUT" + + - name: Upload assets to release (REST, no gh) + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + TAG="${{ steps.tag.outputs.tag }}" + API="https://api.github.com/repos/${GITHUB_REPOSITORY}" + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327) + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") + rid=$(printf '%s' "$rel" | python3 -c 'import json,sys; print(json.load(sys.stdin)["id"])') + for f in "${{ steps.package.outputs.archive }}" "${{ steps.package.outputs.sha }}"; do + aid=$(printf '%s' "$rel" | python3 -c 'import json,sys; n=sys.argv[1]; print(next((a["id"] for a in json.load(sys.stdin)["assets"] if a["name"]==n), ""))' "$f") + [ -n "$aid" ] && curl -sSf -X DELETE -H "Authorization: Bearer $GH_TOKEN" "$API/releases/assets/$aid" > /dev/null + curl -sSf -X POST -H "Authorization: Bearer $GH_TOKEN" -H "Content-Type: application/octet-stream" \ + --data-binary @"$f" "https://uploads.github.com/repos/${GITHUB_REPOSITORY}/releases/$rid/assets?name=$f" \ + | python3 -c 'import json,sys; a=json.load(sys.stdin); print("uploaded", a["name"], a["size"], "bytes")' + done + # The hard requirement, as a check that can fail: all FOUR apr assets (tarball + # sha256) and the eight pv assets must be on the release. A missing one is a red # workflow, never a partial summary. @@ -538,7 +720,7 @@ jobs: # mutation (RED) that this job cannot demonstrate on a real release. verify-apr-assets: name: All four apr assets are on the release - needs: [build, build-apr-cuda, build-apr-cpu] + needs: [build, build-apr-cuda, build-apr-cpu, build-apr-darwin] runs-on: [self-hosted, Linux, X64, clean-room] if: always() steps: @@ -571,6 +753,8 @@ jobs: smoke-cuda: name: smoke apr (cuda) on ${{ matrix.host }} needs: verify-apr-assets + # verify runs under always(); a skipped build two levels up must not skip this + if: ${{ !cancelled() && needs.verify-apr-assets.result == 'success' }} strategy: fail-fast: false matrix: @@ -605,16 +789,25 @@ jobs: T="${{ matrix.target }}" D="$RUNNER_TEMP/apr-cuda-smoke"; rm -rf "$D"; mkdir -p "$D"; cd "$D" API="https://api.github.com/repos/${GITHUB_REPOSITORY}" - rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases/tags/$TAG") + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327), + # and releases/tags/ never returns a draft + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") for f in "apr-${TAG}-${T}-cuda.tar.gz" "apr-${TAG}-${T}-cuda.tar.gz.sha256"; do aid=$(printf '%s' "$rel" | python3 -c 'import json,sys; n=sys.argv[1]; print(next(a["id"] for a in json.load(sys.stdin)["assets"] if a["name"]==n))' "$f") curl -sSfL -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/octet-stream" -o "$f" "$API/releases/assets/$aid" done sha256sum -c "apr-${TAG}-${T}-cuda.tar.gz.sha256" tar xzf "apr-${TAG}-${T}-cuda.tar.gz" - BIN="$D/apr-${TAG}-${T}-cuda/apr" - v=$("$BIN" --version); echo "$v" - case "$v" in *"${TAG#v}"*) ;; *) echo "::error::asset reports '$v', not ${TAG#v}"; exit 1 ;; esac + # A promoted final (#4286) is its rc's exact bytes, so the archive's top directory + # may carry the rc label (apr-vX.Y.Z-rc.N-...). Take the one apr directory it holds. + set -- "$D"/apr-v*-"${T}"-cuda/apr + if [ "$#" -ne 1 ] || [ ! -f "$1" ]; then echo "::error::expected exactly one apr binary in the archive, found: $*"; exit 1; fi + BIN=$1 + v=$("$BIN" --version | head -1); echo "$v" + # #4275: X.Y.Z whole (an rc tag's -rc.N stripped) AND the asset's sha is the tag's commit + COMMIT=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/commits/$TAG" | python3 -c 'import json,sys; print(json.load(sys.stdin)["sha"])') + bash "$GITHUB_WORKSPACE/scripts/release/asset_version_check.sh" "$TAG" "$COMMIT" "$v" || { echo "::error::asset reports '$v', which does not belong to $TAG ($COMMIT)"; exit 1; } [ "$(grep -ac 'libcuda.so' "$BIN")" -ge 1 ] || { echo "::error::downloaded asset is not a CUDA build"; exit 1; } floor=$(objdump -T "$BIN" 2>/dev/null | grep -oE 'GLIBC_[0-9.]+' | sort -Vu | tail -1) echo "highest GLIBC symbol version required: $floor" @@ -633,6 +826,8 @@ jobs: smoke-cpu: name: smoke apr (cpu) on ${{ matrix.host }} needs: verify-apr-assets + # verify runs under always(); a skipped build two levels up must not skip this + if: ${{ !cancelled() && needs.verify-apr-assets.result == 'success' }} env: # The CPU assets build inside rust:1.93.0-bullseye (glibc 2.31) like the cuda lane, so the floor is # declared and GATED: an asset whose bytes import a newer GLIBC_* symbol fails here. 0.66.0 shipped a @@ -672,16 +867,25 @@ jobs: T="${{ matrix.target }}" D="$RUNNER_TEMP/apr-cpu-smoke"; rm -rf "$D"; mkdir -p "$D"; cd "$D" API="https://api.github.com/repos/${GITHUB_REPOSITORY}" - rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases/tags/$TAG") + # by listing, not releases/tags/: an rc is a DRAFT until every fleet host runs it (#4327), + # and releases/tags/ never returns a draft + rel=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/releases?per_page=100" \ + | python3 -c 'import json,sys; t=sys.argv[1]; print(json.dumps(next(r for r in json.load(sys.stdin) if r["tag_name"]==t)))' "$TAG") for f in "apr-${TAG}-${T}-cpu.tar.gz" "apr-${TAG}-${T}-cpu.tar.gz.sha256"; do aid=$(printf '%s' "$rel" | python3 -c 'import json,sys; n=sys.argv[1]; print(next(a["id"] for a in json.load(sys.stdin)["assets"] if a["name"]==n))' "$f") curl -sSfL -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/octet-stream" -o "$f" "$API/releases/assets/$aid" done sha256sum -c "apr-${TAG}-${T}-cpu.tar.gz.sha256" tar xzf "apr-${TAG}-${T}-cpu.tar.gz" - BIN="$D/apr-${TAG}-${T}-cpu/apr" - v=$("$BIN" --version); echo "$v" - case "$v" in *"${TAG#v}"*) ;; *) echo "::error::asset reports '$v', not ${TAG#v}"; exit 1 ;; esac + # A promoted final (#4286) is its rc's exact bytes, so the archive's top directory + # may carry the rc label (apr-vX.Y.Z-rc.N-...). Take the one apr directory it holds. + set -- "$D"/apr-v*-"${T}"-cpu/apr + if [ "$#" -ne 1 ] || [ ! -f "$1" ]; then echo "::error::expected exactly one apr binary in the archive, found: $*"; exit 1; fi + BIN=$1 + v=$("$BIN" --version | head -1); echo "$v" + # #4275: X.Y.Z whole (an rc tag's -rc.N stripped) AND the asset's sha is the tag's commit + COMMIT=$(curl -sSf -H "Authorization: Bearer $GH_TOKEN" -H "Accept: application/vnd.github+json" "$API/commits/$TAG" | python3 -c 'import json,sys; print(json.load(sys.stdin)["sha"])') + bash "$GITHUB_WORKSPACE/scripts/release/asset_version_check.sh" "$TAG" "$COMMIT" "$v" || { echo "::error::asset reports '$v', which does not belong to $TAG ($COMMIT)"; exit 1; } n=$(grep -ac 'libcuda.so' "$BIN" || true) [ "$n" -eq 0 ] || { echo "::error::the downloaded -cpu asset carries $n libcuda.so string(s) — it is a CUDA build"; exit 1; } # The floor is READ on every row and ASSERTED only where the build was @@ -704,7 +908,7 @@ jobs: summary: name: Summary - needs: [build, build-apr-cuda, build-apr-cpu, verify-apr-assets, smoke-cuda, smoke-cpu] + needs: [assets, build, build-apr-cuda, build-apr-cpu, build-apr-darwin, verify-apr-assets, smoke-cuda, smoke-cpu] runs-on: [self-hosted, Linux, X64, clean-room] if: always() steps: @@ -724,6 +928,8 @@ jobs: echo "" if [ "$STATUS" = "success" ]; then echo "- Build matrix: **4/4 targets succeeded**" + elif [ "$STATUS" = "skipped" ] && [ "${{ needs.assets.outputs.present }}" = "true" ]; then + echo "- Build matrix: **not rebuilt** — every asset was already on the release (a promoted rc, #4286)" else echo "- Build matrix: **partial** (status=$STATUS)" fi @@ -734,10 +940,11 @@ jobs: echo " - aarch64-unknown-linux-gnu" echo "" echo "### apr Binary Release: $TAG - four assets, {cuda,cpu} x {x86_64,aarch64}" - echo "- cuda build: ${{ needs.build-apr-cuda.result }} · cpu build: ${{ needs.build-apr-cpu.result }}" + echo "- cuda build: ${{ needs.build-apr-cuda.result }} · cpu build: ${{ needs.build-apr-cpu.result }} · darwin build (mini): ${{ needs.build-apr-darwin.result }}" echo "- all four on the release: ${{ needs.verify-apr-assets.result }} · smoke cuda (yoga+gx10): ${{ needs.smoke-cuda.result }} · smoke cpu (clean-room+yoga+gx10): ${{ needs.smoke-cpu.result }}" echo " - apr-$TAG-x86_64-unknown-linux-gnu-cuda.tar.gz (+ .sha256)" echo " - apr-$TAG-aarch64-unknown-linux-gnu-cuda.tar.gz (+ .sha256)" echo " - apr-$TAG-x86_64-unknown-linux-gnu-cpu.tar.gz (+ .sha256)" echo " - apr-$TAG-aarch64-unknown-linux-gnu-cpu.tar.gz (+ .sha256)" + echo " - apr-$TAG-aarch64-apple-darwin-cpu.tar.gz (+ .sha256)" } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/book-contracts.yml b/.github/workflows/book-contracts.yml index e1ca6e9c65..b28d4404fe 100644 --- a/.github/workflows/book-contracts.yml +++ b/.github/workflows/book-contracts.yml @@ -39,6 +39,15 @@ on: schedule: - cron: '45 22 * * *' + +# #3676: no concurrency group meant every superseded PR push ran the ~18 min +# chapter suite to completion. Sampled 24 h to 2026-09-24 19:15Z: ~1650 of ~3700 +# wall-minutes of this workflow's pull_request runs came after a newer push +# to the same PR had already started. Same shape as ci.yml: one group per PR, a +# new push cancels the old run, and push/schedule runs are never cancelled. +concurrency: + group: book-contracts-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: contents: read diff --git a/.github/workflows/book.yml b/.github/workflows/book.yml index ba40bbd70b..bec305205e 100644 --- a/.github/workflows/book.yml +++ b/.github/workflows/book.yml @@ -60,9 +60,14 @@ permissions: pages: write id-token: write +# #3676: one group per PR, and a newer push cancels the superseded run. The old +# workflow-level group "pages" (cancel false) held EVERY run -- all PRs, main and +# the nightly -- in one global queue, so each PR's mdBook check waited behind +# every other PR's. Only the Pages deploy needs that serialisation, so the +# "pages" group now sits on the deploy job alone. Non-PR events never cancel. concurrency: - group: "pages" - cancel-in-progress: false + group: book-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} jobs: build: @@ -203,6 +208,10 @@ jobs: runs-on: [self-hosted, Linux, X64, clean-room] needs: build if: github.event_name == 'push' && github.ref == 'refs/heads/main' + # GitHub Pages deploys never overlap and are never cancelled (was workflow-level). + concurrency: + group: "pages" + cancel-in-progress: false permissions: pages: write id-token: write diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cfc9e1c366..4572542af7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -388,6 +388,16 @@ jobs: } parent="/mnt/nvme-raid0/targets/aprender-ci/${PR_OR_REF}" reg="/mnt/nvme-raid0/cargo-ci/registry/${PR_OR_REF}" + # infra#1059: admit this target unit against gx10's disk budget BEFORE creating it + # (free >= 50G + 25G x (live+1), live < 2; waits <=20m, then exit 1). The admit + # record is what the runner's JOB_COMPLETED hook reaps, and nothing else. Hosts + # without the gate skip; gx10 exports CI_TARGET_ADMIT_REQUIRED=1, so a missing + # gate there is red, never a silent skip. + if command -v ci-target-admit >/dev/null 2>&1; then + ci-target-admit "${parent}/run-${GITHUB_RUN_ID}" || exit 1 + elif [ "${CI_TARGET_ADMIT_REQUIRED:-0}" = 1 ]; then + echo "::error::infra#1059: ci-target-admit is required on ${RUNNER_NAME:-this runner} and is missing"; exit 1 + fi heal_path /mnt/nvme-raid0/targets/aprender-ci "${parent}/run-${GITHUB_RUN_ID}" heal_path /mnt/nvme-raid0/cargo-ci/registry "$reg" ls -ld "$parent" "${parent}/run-${GITHUB_RUN_ID}" "$reg" @@ -891,11 +901,17 @@ jobs: # workspace. Runner (noah:1000) can't git-clean them on next run # → checkout fails → CI breaks. This runs inside the container # (as root) restoring host ownership for subsequent bare-metal jobs. + # infra#1059: mount the target unit only if it exists. When the disk + # admit gate refused it, a bind mount here would make docker create the + # refused unit as root, unrecorded, so the reap hook would never free it. + unit="/mnt/nvme-raid0/targets/aprender-ci/${PR_OR_REF}/run-${GITHUB_RUN_ID}" + target_mount=() + if [ -d "$unit" ]; then target_mount=(-v "${unit}:/workspace/target"); fi docker run --rm \ -e CI -e GITHUB_ACTIONS -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -e GITHUB_EVENT_NAME -e GITHUB_WORKFLOW \ -v "${GITHUB_WORKSPACE}:/workspace" \ -v "/mnt/nvme-raid0/cargo-ci/registry/${PR_OR_REF}:/usr/local/cargo/registry" \ - -v "/mnt/nvme-raid0/targets/aprender-ci/${PR_OR_REF}/run-${GITHUB_RUN_ID}:/workspace/target" \ + "${target_mount[@]}" \ "$IMAGE" \ bash -c 'chown -R 1000:1000 /workspace || true; chown -R 1000:1000 /usr/local/cargo/registry || true; chown -R 1000:1000 /workspace/target || true' @@ -958,6 +974,43 @@ jobs: # discriminating runner label, so a bare [self-hosted, X64, Linux] selector # can never again silently land a job on a GPU/dev runner without the # sovereign-ci registry. Pure text check; runs on the clean-room pool. + # #3668: ONE docs-only decision for the run. A roadmap-only PR paid ~26 min of + # guard-cargo and ~6 of determinism for a diff neither reads. This job asks + # scripts/ci_docs_only.sh (which asks ci_test_tier.sh's #3658 definition: + # every path under docs/roadmaps/ or docs/audits/ and present at HEAD) and + # publishes docs_only. The consumers are written FAIL-CLOSED: they read + # `!cancelled() && needs.changes.outputs.docs_only != 'true'`, so this job + # being skipped (every non-PR event), failing, or printing anything but + # `true` runs everything. `gate` accepts their `skipped` only when this job + # SUCCEEDED and said true (GATE-DOCS-ONLY-RULE, + # scripts/check_ci_gate_docs_only_rule.sh). + changes: + runs-on: [self-hosted, Linux, clean-room] # arch-neutral: intel, yoga or gx10 (#3100) + timeout-minutes: 10 # shallow checkout + one git diff + one script + if: github.event_name == 'pull_request' + outputs: + docs_only: ${{ steps.decide.outputs.docs_only }} + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 1 + persist-credentials: false + - name: Is this PR docs-only? (#3668) + id: decide + run: | + set -euo pipefail + git fetch --no-tags --depth=1 origin \ + "+refs/heads/${GITHUB_BASE_REF}:refs/remotes/origin/${GITHUB_BASE_REF}" + # --no-renames (#3664): a source moved into docs/ must list BOTH paths. + git diff --no-renames --name-only "origin/${GITHUB_BASE_REF}" HEAD > "$RUNNER_TEMP/touched.txt" + printf 'touched (tip vs tip): %s path(s)\n' "$(wc -l < "$RUNNER_TEMP/touched.txt" | tr -d ' ')" + bash scripts/ci_docs_only.sh --event "${GITHUB_EVENT_NAME}" --diff-from "$RUNNER_TEMP/touched.txt" \ + > "$RUNNER_TEMP/docs_only.txt" + cat "$RUNNER_TEMP/docs_only.txt" + grep -E '^docs_only=(true|false)$' "$RUNNER_TEMP/docs_only.txt" >> "$GITHUB_OUTPUT" + - name: The docs-only decision can still go RED (case table) + run: bash scripts/ci_docs_only.sh --self-test + guard-tree: runs-on: [self-hosted, Linux, clean-room] # arch-neutral: intel, yoga or gx10 (#3100) # 20: the operator's hard PR budget (rule E, 2026-09-10), and it OVERRIDES @@ -1094,6 +1147,70 @@ jobs: env: GH_TOKEN: ${{ github.token }} run: bash scripts/guard_tree.sh --no-cargo + # #3668: these three ran in guard-cargo, but none of them invokes cargo -- + # guard_tree.sh --no-cargo files them as cargo guards only because their + # SOURCE says `cargo doc` / `cargo install` in a comment or a pattern. They + # are the guard-cargo steps whose reads include docs/roadmaps/ or docs/audits/ + # (docs/**/*.md, a `git ls-files | grep dogfood.sh$`), so they live here: + # guard-cargo is skipped on a docs-only PR (the `changes` job) and guard-tree + # never is, so a docs-only diff is still judged by every guard that reads it. + # One runner, one prose, and the user-scope copy stays a <=70-line shim + # that fails CLOSED. The cap lives here, in the repo on protected main; + # the file it caps lives at user scope, where it could otherwise raise its + # own ceiling. + - name: One dogfood runner, and the user-scope copy is a gated shim + run: bash scripts/check_dogfood_shim.sh + # §4.1 names BOTH of these in the merge-phase blocking set, alongside the + # fabricated-baseline guard above. They shipped unwired, which is why the + # five `2.93x Ollama` claims in book/ and the four deleted harnesses were + # each caught by a person rather than by CI. + # + # Neither is baselined as an exemption, deliberately. The tip commit of + # this epic says PERF-009 is "enforced by a shrink-only guard"; a guard + # named by no workflow enforces nothing, and recording that in a baseline + # file would make the claim true only in the sense that the gap is + # written down. + # + # This step is ALSO the I-12 `[X]`-figure gate, which v2.2 §5 listed as + # "not written" against this file. It was right: the comparator pattern + # required the competitor's name to sit immediately after the ratio, in + # ASCII, so of the six shapes §0.1 names it caught one. PERF-010 admits + # the connector ("36.9x over FasterTransformer") and U+00D7, and adds + # placeholders bound to a unit ("XX tok/s"). Twelve pre-existing sites + # surfaced and are baselined shrink-only; one was a LIVE comparator claim + # in book/ and is deleted. + - name: No claim literals on user-facing surfaces + run: bash scripts/check_no_claim_literals.sh + - name: No claim-literal guard regression (case table) + run: bash scripts/check_no_claim_literals.sh --selftest + # PERF-010, the receipt rule's namesake check. The guard above is the + # NEGATIVE half -- a comparator ratio may not appear at all. This is the + # POSITIVE half over the class that is allowed to stay: a speed comparison + # naming no competitor is legal iff it cites the evidence/ receipt that + # produced it (v2.2 section 3.2). Text-only and deterministic, so it + # belongs in the merge phase (section 4.1) rather than at release. + # + # It found 146 uncited speed comparisons across 30 files and ZERO citing a + # receipt -- a 0% citation rate on the very number class this epic is + # named for. Baselined shrink-only; the count is the finding. + - name: Performance claims cite their receipts (PERF-010) + run: bash scripts/check_perf_claims_cite_receipts.sh + - name: Claims-cite-receipts guard can still go RED (case table) + run: bash scripts/check_perf_claims_cite_receipts.sh --selftest + # The gate's two skip rules, executed from ci.yml's own marker blocks. Neither + # was run by any step: guard_tree.sh files both as cargo guards (their text + # names guard-cargo) and no workflow dispatches --cargo-only, so the mutants + # rule (#3676) passed check_guards_are_wired.sh on a comment mention alone. + # They read only ci.yml and invoke no build tool, so they run here -- the job + # that runs on every PR, including the one that edits the gate (#3668). + - name: gate accepts a skipped mutants job only when that is its filter (#3676) + run: bash scripts/check_ci_gate_mutants_rule.sh + - name: gate mutants rule can still go RED (#3676) + run: bash scripts/check_ci_gate_mutants_rule.sh --self-test + - name: gate accepts a skipped guard-cargo/determinism only on a docs-only PR (#3668) + run: bash scripts/check_ci_gate_docs_only_rule.sh + - name: gate docs-only rule can still go RED (#3668) + run: bash scripts/check_ci_gate_docs_only_rule.sh --self-test # scripts/tests/guard_tree_test.sh is guard_tree.sh's OWN case table # (BSE-01, PMAT-1062) -- it proves guard_tree.sh's no-fail-fast # behaviour against a fixture, proves a fail-fast MUTANT of it goes RED @@ -1112,6 +1229,14 @@ jobs: # see scripts/check_no_timing_in_required.sh). - name: "guard_tree.sh's parallel dispatcher case table (PMAT-1098)" run: bash scripts/tests/guard_tree_parallel_test.sh + # PVL-001 EV-4 (aprender#4168): `make contracts` propagates pv's exit. + # Under .ONESHELL a recipe's status is its LAST line's, so a failing + # `pv lint | tail -5` exited 0. The table runs the recipes EXTRACTED from + # the Makefile against a fake pv (one planted failure per row), then + # removes each mechanism of the fix and asserts its row goes RED. + # Cargo-free: pv, cargo and the sibling scripts are stubs. + - name: "make contracts propagates pv's exit (PVL-001 EV-4, aprender#4168)" + run: bash scripts/tests/make_contracts_propagates.sh # Fleet hygiene case tables (docs/specifications/ci-fleet-hygiene.md §6): the queue # steward, the packing/history ledgers, the tier ledger and the roadmap resolve-dirty # runner each ship a hermetic --selftest; check_guards_are_wired.sh refuses a script @@ -1396,11 +1521,13 @@ jobs: # 0.67.0 reconcile found 48 merged PRs since the previous tag of which only # 9 closed anything; that is what this guard exists to stop. # - # THE BODY ARRIVES THROUGH `env:`, NEVER INTERPOLATED INTO THE SCRIPT. - # `${{ }}` is substituted textually BEFORE bash sees the line, so a body - # containing a backtick or $(...) would execute on the runner -- and this - # body is attacker-controlled by construction. `env:` passes it as data. - # scripts/check_workflow_env_defined.sh enforces the same rule. + # THE BODY IS NEVER INTERPOLATED INTO THE SCRIPT. `${{ }}` is substituted + # textually BEFORE bash sees the line, so a body containing a backtick or + # $(...) would execute on the runner -- and this body is attacker-controlled + # by construction. Since #3298 it is fetched live by `gh api` straight into + # a file (it used to arrive through `env:` from the stale event payload); + # either way it is data. scripts/check_workflow_env_defined.sh enforces + # the env: side of that rule. # # pull_request only: on merge_group there is no pull_request payload, and # a body is fixed before a PR is ever enqueued. @@ -1414,12 +1541,19 @@ jobs: # so no guard result can be its precondition. (#3282 classifies the other # 46, which is a judgment call per step rather than one blanket edit.) if: "!cancelled() && github.event_name == 'pull_request'" + # #3298: the body is read LIVE from the API, not from the event payload. + # `github.event.pull_request.body` is frozen when the event is minted and + # `gh run rerun` replays that payload, so fixing the body per the guard's + # own FAIL message could never be re-graded by a rerun -- only by a push. + # Still data, never interpolated: gh writes it straight to the file. env: - PR_BODY: ${{ github.event.pull_request.body }} + PR_NUMBER: ${{ github.event.pull_request.number }} + GH_TOKEN: ${{ github.token }} run: | set -euo pipefail bash scripts/check_pr_closes_issue.sh --self-test - printf '%s' "$PR_BODY" > "$RUNNER_TEMP/pr-body.txt" + gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \ + --jq '.body // ""' > "$RUNNER_TEMP/pr-body.txt" bash scripts/check_pr_closes_issue.sh --body "$RUNNER_TEMP/pr-body.txt" # APR-RELEASE-001 §11.1, WIRED. The section shipped as prose in #3268 and # nothing read it: `grep -rl ont-delta scripts/ .github/ Makefile` returned @@ -1428,9 +1562,9 @@ jobs: # and whose only mechanism IS a review comment is this repo's anti-theater # class, one level up from the guard it sits beside. # - # Same body-handling rule as the step above: the body arrives through - # `env:` as DATA and is never interpolated into the script. It is - # attacker-controlled by construction. + # Same body-handling rule as the step above: the body is fetched live by + # `gh api` into a file as DATA and is never interpolated into the script. + # It is attacker-controlled by construction. # # The changed-file list is the other input, and it is what makes the guard # non-trivial: half the sweep surface ("a known-red list anywhere") is @@ -1447,13 +1581,16 @@ jobs: # so no guard result can be its precondition. (#3282 classifies the other # 46, which is a judgment call per step rather than one blanket edit.) if: "!cancelled() && github.event_name == 'pull_request'" + # #3298: body and file list now come from the same place (the live API) + # at the same time. The body used to be the event SNAPSHOT while the file + # list was live, so a rerun after a body edit re-graded the OLD body. env: - PR_BODY: ${{ github.event.pull_request.body }} PR_NUMBER: ${{ github.event.pull_request.number }} GH_TOKEN: ${{ github.token }} run: | set -euo pipefail - printf '%s' "$PR_BODY" > "$RUNNER_TEMP/ont-body.txt" + gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \ + --jq '.body // ""' > "$RUNNER_TEMP/ont-body.txt" gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files" --paginate \ --jq '.[].filename' > "$RUNNER_TEMP/ont-files.txt" bash scripts/check_pr_ont_delta.sh \ @@ -1501,6 +1638,11 @@ jobs: # minutes onto every green run for no ordering that anything requires. guard-cargo: runs-on: [self-hosted, Linux, clean-room] # arch-neutral: intel, yoga or gx10 (#3100) + # #3668: skipped on a docs-only PR -- no step here reads docs/roadmaps/ or + # docs/audits/ (the three that did moved to guard-tree). Fail-closed: a + # skipped/failed `changes` job leaves docs_only empty, and this runs. + needs: [changes] + if: ${{ !cancelled() && needs.changes.outputs.docs_only != 'true' }} # 20: the operator's hard PR budget (rule E, 2026-09-10), and it OVERRIDES # the BSE-05 formula rather than being derived from it. # @@ -1620,6 +1762,16 @@ jobs: done mkdir -p "$path" } + # infra#1059: admit this target unit against gx10's disk budget BEFORE creating it + # (free >= 50G + 25G x (live+1), live < 2; waits <=20m, then exit 1). The admit + # record is what the runner's JOB_COMPLETED hook reaps, and nothing else. Hosts + # without the gate skip; gx10 exports CI_TARGET_ADMIT_REQUIRED=1, so a missing + # gate there is red, never a silent skip. + if command -v ci-target-admit >/dev/null 2>&1; then + CI_TARGET_ADMIT_WAIT_S=300 ci-target-admit "$GUARD_TARGET_DIR" || exit 1 # 5 min, not 20: the 30 min job budget is rule E and is never widened; gx10 guard-cargo max 23 min (46 runs, 2026-09-24) + elif [ "${CI_TARGET_ADMIT_REQUIRED:-0}" = 1 ]; then + echo "::error::infra#1059: ci-target-admit is required on ${RUNNER_NAME:-this runner} and is missing"; exit 1 + fi heal_path /mnt/nvme-raid0/targets/aprender-ci "$GUARD_TARGET_DIR" heal_path /mnt/nvme-raid0/cargo-ci/home "$GUARD_CARGO_HOME/registry" printf 'target dir: %s\n' "$GUARD_TARGET_DIR" @@ -1671,12 +1823,6 @@ jobs: # It found `pmat analyze reachability` bare in BOTH copies on its first run. - name: The release runner must not resolve a verifier through PATH run: bash scripts/check_verifier_pinning.sh - # One runner, one prose, and the user-scope copy stays a <=70-line shim - # that fails CLOSED. The cap lives here, in the repo on protected main; - # the file it caps lives at user scope, where it could otherwise raise its - # own ceiling. - - name: One dogfood runner, and the user-scope copy is a gated shim - run: bash scripts/check_dogfood_shim.sh # deny.toml is 7.2 KB of licence / banned-crate / source-allowlist policy # that ran in NO workflow -- only `make deny`, which is not a prerequisite # of any tier, so nothing but a human typing it ever invoked it. Meanwhile @@ -2007,43 +2153,6 @@ jobs: # ollama never invoked (#2679). - name: No fabricated comparator baselines run: bash scripts/check_no_fabricated_baselines.sh - # §4.1 names BOTH of these in the merge-phase blocking set, alongside the - # fabricated-baseline guard above. They shipped unwired, which is why the - # five `2.93x Ollama` claims in book/ and the four deleted harnesses were - # each caught by a person rather than by CI. - # - # Neither is baselined as an exemption, deliberately. The tip commit of - # this epic says PERF-009 is "enforced by a shrink-only guard"; a guard - # named by no workflow enforces nothing, and recording that in a baseline - # file would make the claim true only in the sense that the gap is - # written down. - # - # This step is ALSO the I-12 `[X]`-figure gate, which v2.2 §5 listed as - # "not written" against this file. It was right: the comparator pattern - # required the competitor's name to sit immediately after the ratio, in - # ASCII, so of the six shapes §0.1 names it caught one. PERF-010 admits - # the connector ("36.9x over FasterTransformer") and U+00D7, and adds - # placeholders bound to a unit ("XX tok/s"). Twelve pre-existing sites - # surfaced and are baselined shrink-only; one was a LIVE comparator claim - # in book/ and is deleted. - - name: No claim literals on user-facing surfaces - run: bash scripts/check_no_claim_literals.sh - - name: No claim-literal guard regression (case table) - run: bash scripts/check_no_claim_literals.sh --selftest - # PERF-010, the receipt rule's namesake check. The guard above is the - # NEGATIVE half -- a comparator ratio may not appear at all. This is the - # POSITIVE half over the class that is allowed to stay: a speed comparison - # naming no competitor is legal iff it cites the evidence/ receipt that - # produced it (v2.2 section 3.2). Text-only and deterministic, so it - # belongs in the merge phase (section 4.1) rather than at release. - # - # It found 146 uncited speed comparisons across 30 files and ZERO citing a - # receipt -- a 0% citation rate on the very number class this epic is - # named for. Baselined shrink-only; the count is the finding. - - name: Performance claims cite their receipts (PERF-010) - run: bash scripts/check_perf_claims_cite_receipts.sh - - name: Claims-cite-receipts guard can still go RED (case table) - run: bash scripts/check_perf_claims_cite_receipts.sh --selftest # The single source of truth needs a truth check. An authorless policy # number, a `inherited_from` pointing at a document with no history in any # ref, a compute_class no build in this tree reaches, and a cell exception @@ -2799,6 +2908,7 @@ jobs: outputs: gpu_touched: ${{ steps.decide.outputs.gpu_touched }} gpu_crates: ${{ steps.decide.outputs.gpu_crates }} + cuda_lint: ${{ steps.decide.outputs.cuda_lint }} steps: - uses: actions/checkout@v7 with: @@ -2829,7 +2939,7 @@ jobs: bash scripts/ci_gpu_touched.sh --diff-from "$RUNNER_TEMP/touched.txt" \ > "$RUNNER_TEMP/gpu.txt" cat "$RUNNER_TEMP/gpu.txt" - grep -E '^(gpu_touched|gpu_crates|reason)=' "$RUNNER_TEMP/gpu.txt" >> "$GITHUB_OUTPUT" + grep -E '^(gpu_touched|gpu_crates|cuda_lint|reason)=' "$RUNNER_TEMP/gpu.txt" >> "$GITHUB_OUTPUT" # ROW 67-C1 — `gpu-quick` on gx10 (GB10, sm_121, aarch64). # Promotion to required: row 68-C3, after 5 consecutive green gx10 nightlies. @@ -2925,14 +3035,20 @@ jobs: # (the yoga nightly axis is #3060 / YOGA-NIGHTLY-001). cuda-unit: needs: [gpu-touched] - if: github.event_name == 'pull_request' && needs.gpu-touched.outputs.gpu_touched == '1' + # gpu_touched=1 runs every step. cuda_lint=1 alone (an apr-cli-only diff, + # #4336) runs ONLY the clippy step: the test steps carry their own + # gpu_touched `if:`, so yoga is held for a check build, not the test builds. + if: github.event_name == 'pull_request' && (needs.gpu-touched.outputs.gpu_touched == '1' || needs.gpu-touched.outputs.cuda_lint == '1') runs-on: [self-hosted, Linux, X64, cuda, yoga, ephemeral, docker] # 15 -> 35 (#3810): the derived cuda-only set needs a second, default-feature # test build of aprender-serve to list against. MEASURED on this PR's own run # (job 107295428985, yoga-eph, sharing yoga with a coverage job): the step # took 18 m 20 s and the job 21 m 18 s (tests alone 51 s). 25 left 3.7 min # of margin on a shared runner, a future flaky timeout, so 35. - timeout-minutes: 35 + # 35 -> 50 (#3636): the clippy step is a check-profile build of the cuda tree, + # separate from the release test builds. Not yet measured on yoga; 15 min is a + # ceiling, not an estimate. Re-derive from the first run and tighten. + timeout-minutes: 50 continue-on-error: false concurrency: group: perf-yoga @@ -2950,6 +3066,7 @@ jobs: # (operator, 2026-09-09), not a training host. Nothing to yield to. The # `perf-yoga` concurrency group above is what keeps two runs off it. - name: aprender-gpu cuda unit tests (--features cuda --lib --release) + if: needs.gpu-touched.outputs.gpu_touched == '1' run: | nice -n 19 cargo test -p aprender-gpu --features cuda --lib --release # THE SERVE CUDA SET IS DERIVED, NOT FILTERED (#3810). The step used to filter @@ -2985,6 +3102,7 @@ jobs: # counts as a pass. The grep is UNANCHORED, because under --nocapture the # message lands on the `test X ... ` line. - name: "aprender-serve cuda unit tests (the derived cuda-only set, by name)" + if: needs.gpu-touched.outputs.gpu_touched == '1' run: | set -euo pipefail export LC_ALL=C @@ -3034,6 +3152,19 @@ jobs: echo "$summary" printf 'cuda-only: %s passed + %s ignored = %s derived, 0 device skips, %s model-absent skip line(s)\n' \ "$passed" "$ignored" "$n_only" "${model_skips:-0}" + # #3636: `clippy --features cuda` was never run by any gate, and 62 findings + # piled up in cuda-gated code (compute, train, serve) before anyone looked. + # This is the only job with nvcc, so the lint lives here. The script runs its + # own planted-mutant self-test first: an unused import under + # #[cfg(feature = "cuda")] must turn it RED, or the gate is refused as vacuous. + # No step `if:`: this step runs whenever the job does, including on an + # apr-cli-only diff (cuda_lint=1, gpu_touched=0) — apr-cli has 30 files of + # #[cfg(feature = "cuda")] code that only this lint builds (#4336). + - name: "clippy --features cuda is clean (#3636)" + run: | + set -euo pipefail + bash scripts/check_clippy_cuda.sh --self-test + nice -n 19 bash scripts/check_clippy_cuda.sh # APEX-001 EV-2a: the same figure, rendered on two architectures, must be the same bytes. # @@ -3047,6 +3178,10 @@ jobs: # font, a locale or a cached artefact from a previous job and quietly agree for the wrong reason. determinism: name: determinism (${{ matrix.arch }}) + # #3668: a docs-only PR renders nothing new; determinism-compare is skipped + # with it, and gate accepts that only under docs_only=true. + needs: [changes] + if: ${{ !cancelled() && needs.changes.outputs.docs_only != 'true' }} strategy: fail-fast: false # if one arch fails we still want the other's receipt to compare against matrix: @@ -3184,6 +3319,123 @@ jobs: path: target/aprender-viz/raster-receipt.json if-no-files-found: error + # ── PVL-001 EV-9 (#4083): the provable ladder, ADVISORY ──────────────────── + # + # `pv discharge run` over the Lean tree: build.sh (Mathlib cache get, and a cache + # MISS declines instead of silently building Mathlib), then `check` with every arm + # (--strict, the comparator, --leanchecker), then the tracked discharge-summary.json. + # The committed summary must be what the run writes (summary-fresh: the tree carries + # the summary and CI regenerates it and diffs it; a gate never rewrites its own + # baseline, PVL-001 §5.3). + # + # ADVISORY: NOT in `gate`'s `needs:` (cop ruling, 2026-09-24). Measured at 4a870daf5 on + # lambda (4m21s, 1.4 GB RSS), the ladder is RED: 7 escape-allowlist.yaml entries are still + # `confirmed_by: pending`, and only a person may confirm them (§5.3). Flip to required -- + # add it to `gate.needs` and read its result there -- only when that is clear. + # + # The clean-room X64 pool is intel-clean-room-* plus yoga-build, yoga-build2 and yoga-build3, + # and infra-8d verified elan and a writable /mnt/nvme-raid0/lean-cache on every one of them + # (2026-09-25). A host added to the pool without them DECLINES here, never builds cold. + # X64 is mandatory: `ci / lint` once landed on aarch64 gx10 and failed on x86-only code. + # 90 min: under the merge queue's 180 min. + # build.sh is ~6 min on a warm cache, and the leanchecker arm's own cap is 3600 s. + provable-ladder: + runs-on: [self-hosted, Linux, clean-room, X64] + timeout-minutes: 90 + env: + LEAN: crates/aprender-contracts-staging/lean + SUMMARY: crates/aprender-contracts-staging/discharge-summary.json + ELAN_HOME: /home/noah/.elan + LEAN_CACHE_ROOT: /mnt/nvme-raid0/lean-cache + CARGO_TARGET_DIR: /mnt/nvme-raid0/targets/aprender-ci/${{ github.event.pull_request.number || github.ref_name }}/provable-ladder + PV: /mnt/nvme-raid0/targets/aprender-ci/${{ github.event.pull_request.number || github.ref_name }}/provable-ladder/debug/pv + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 1 + + - name: Preconditions -- elan, the lean cache, and the files the ladder judges + id: pre + # A missing precondition DECLINES (rc 2); it is never a silent cold build. The toolchain + # is read from lean-toolchain, so this workflow never names a Lean version. The .lake dir + # lives in the cache, keyed on lean-toolchain + lake-manifest.json. + shell: bash + run: | + set -euo pipefail + decline() { echo "decline: $1" | tee -a "$GITHUB_STEP_SUMMARY"; exit 2; } + export PATH="$ELAN_HOME/bin:$PATH" + command -v elan >/dev/null || decline "no elan in $ELAN_HOME on $(hostname) (proof-toolchain, infra)" + [ -d "$LEAN_CACHE_ROOT" ] || decline "no $LEAN_CACHE_ROOT on $(hostname) (proof-toolchain, infra)" + for f in "$LEAN/lean-toolchain" "$LEAN/lake-manifest.json" "$LEAN/build.sh"; do + [ -f "$f" ] || decline "$f is not in this tree" + done + # summary-fresh diffs a TRACKED file: `git diff` exits 0 on an untracked path. + git ls-files --error-unmatch -- "$SUMMARY" >/dev/null 2>&1 || decline "$SUMMARY is not tracked" + key=$(cat "$LEAN/lean-toolchain" "$LEAN/lake-manifest.json" | sha256sum | cut -c1-16) + cache=$LEAN_CACHE_ROOT/$key + mkdir -p "$cache" + flock "$cache.lock" elan toolchain install "$(cat "$LEAN/lean-toolchain")" + # A real .lake dir would make `ln` write the link INSIDE it instead of replacing it. + [ ! -e "$LEAN/.lake" ] || [ -L "$LEAN/.lake" ] || decline "$LEAN/.lake is a real directory, not the cache link" + ln -sfn "$cache" "$LEAN/.lake" + echo "$ELAN_HOME/bin" >> "$GITHUB_PATH" + echo "cache=$cache" >> "$GITHUB_OUTPUT" + + - name: Build the in-tree pv + id: build + shell: bash + run: | + set -euo pipefail + cargo build --locked -p aprender-contracts-cli --bin pv + test -x "$PV" + + - name: pv discharge run (build.sh, then check --strict + comparator + leanchecker) + id: discharge + shell: bash + # One run at a time per cache: two runs unpacking into the same .lake would race. + # set +e: Actions runs `bash -e`, which would abort at a failing run before its rc and + # its log tail are recorded. + run: | + set +e + flock "${{ steps.pre.outputs.cache }}.lock" "$PV" discharge run "$LEAN" > "$RUNNER_TEMP/discharge-run.log" 2>&1 + rc=$? + tail -n 40 "$RUNNER_TEMP/discharge-run.log" + echo "pv discharge run: rc=$rc" >> "$GITHUB_STEP_SUMMARY" + exit "$rc" + + - name: summary-fresh -- the committed discharge-summary.json is what the run wrote + # Only when the run RAN (it writes the summary on failure too). Skipped means nothing + # was regenerated, and a diff of an unregenerated file would pass vacuously. + if: ${{ !cancelled() && (steps.discharge.outcome == 'success' || steps.discharge.outcome == 'failure') }} + shell: bash + run: git diff --exit-code -- "$SUMMARY" + + - name: pv lint contracts/ + if: ${{ !cancelled() && steps.build.outcome == 'success' }} + shell: bash + run: '"$PV" lint contracts/' + + - name: Upload the discharge logs + if: ${{ !cancelled() }} + uses: actions/upload-artifact@v7 + with: + name: provable-ladder + path: | + ${{ runner.temp }}/discharge-run.log + crates/aprender-contracts-staging/lean/discharge.json + crates/aprender-contracts-staging/discharge-summary.json + if-no-files-found: warn + + - name: Advisory verdict + if: always() + shell: bash + run: | + { + echo "### provable-ladder (PVL-001 EV-9, #4083): ADVISORY, not in gate.needs" + echo "Job status: ${{ job.status }}. It becomes required only when this blocker is clear:" + echo "1. escape-allowlist.yaml: 7 entries are \`confirmed_by: pending\`, and a person must confirm them or delete the axioms (PVL-001 §5.3)" + } >> "$GITHUB_STEP_SUMMARY" + gate: runs-on: [self-hosted, Linux, clean-room] # arch-neutral: intel, yoga or gx10 (#3100) # 22: BSE-05 formula T = max(15, ceil(1.5*p99), p99+20) over this job's own @@ -3192,7 +3444,7 @@ jobs: # `needs.*.result`; the 20-minute term is the headroom BSE-05 adds so a # slow runner is never the thing that fails a required check. timeout-minutes: 22 - needs: [ci, workspace-test, mutants, guard-tree, guard-cargo, determinism-compare] + needs: [ci, workspace-test, mutants, guard-tree, guard-cargo, determinism-compare, changes] if: always() steps: - name: Check required jobs @@ -3201,13 +3453,6 @@ jobs: echo "guard-tree failed: ${{ needs.guard-tree.result }}" exit 1 fi - # guard-cargo is the cargo half of the same guard universe (BSE-01). - # It is judged exactly like guard-tree: a required check that - # is not read by `gate` is not a required check. - if [ "${{ needs.guard-cargo.result }}" != "success" ]; then - echo "guard-cargo failed: ${{ needs.guard-cargo.result }}" - exit 1 - fi if [ "${{ needs.ci.result }}" != "success" ]; then echo "ci failed: ${{ needs.ci.result }}" exit 1 @@ -3216,12 +3461,29 @@ jobs: echo "workspace-test failed: ${{ needs.workspace-test.result }}" exit 1 fi - # APEX-001 EV-2a: the two-architecture determinism comparison. A required check that - # `gate` does not read is not a required check, so it is read here like guard-tree. - if [ "${{ needs.determinism-compare.result }}" != "success" ]; then - echo "determinism-compare failed: ${{ needs.determinism-compare.result }}" - exit 1 - fi + # guard-cargo is the cargo half of the same guard universe (BSE-01), and + # determinism-compare is APEX-001 EV-2a's two-architecture comparison. A + # required check that `gate` does not read is not a required check, so both + # are read here -- and each may be `skipped` ONLY on a docs-only PR (#3668). + DOCS="${{ needs.changes.outputs.docs_only }}" + CHG="${{ needs.changes.result }}" + GC="${{ needs.guard-cargo.result }}" + DC="${{ needs.determinism-compare.result }}" + # --- GATE-DOCS-ONLY-RULE-BEGIN (#3668; scripts/check_ci_gate_docs_only_rule.sh runs this block) --- + # A skip is the docs-only skip only when the `changes` job SUCCEEDED and + # said docs_only=true. Any other skip -- changes skipped, failed, or said + # false -- means a job that should have run did not, and that is a failure. + skip_ok=0 + if [ "$CHG" = success ] && [ "$DOCS" = true ]; then skip_ok=1; fi + for pair in "guard-cargo=$GC" "determinism-compare=$DC"; do + job=${pair%%=*}; res=${pair#*=} + case "$res:$skip_ok" in + success:*) echo "$job: success" ;; + skipped:1) echo "$job: skipped -- docs-only PR (changes: docs_only=true, #3668)" ;; + *) echo "$job failed: $res (changes=$CHG docs_only='$DOCS'; a skip passes only on a docs-only PR)"; exit 1 ;; + esac + done + # --- GATE-DOCS-ONLY-RULE-END --- # Diff-scoped mutation gate (PMAT gap #1): blocking on PRs. # `skipped` is the expected result on push-to-main (the job has # `if: github.event_name == 'pull_request'`); treat it as pass so diff --git a/.github/workflows/install-script.yml b/.github/workflows/install-script.yml index df4b38dae6..7b5110b5d8 100644 --- a/.github/workflows/install-script.yml +++ b/.github/workflows/install-script.yml @@ -37,6 +37,12 @@ on: schedule: - cron: '0 23 * * *' + +# #3676: a newer push to the same PR cancels the superseded run +# (scripts/check_pr_workflows_cancel_superseded.sh). +concurrency: + group: install-script-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: contents: read diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 5b7937e025..88d1152d5d 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -12,9 +12,9 @@ # job runs on the fleet. x86_64 builds on yoga and aarch64 natively on gx10, both # inside rust:1.93.0-bullseye (binary-release.yml's CPU-lane pattern), so the # nightly's glibc floor is 2.31; the ubuntu-latest build needed GLIBC_2.39 and did -# not run on lambda's 22.04. The macOS and Windows targets are gone: no self-hosted -# macOS or Windows runner is registered in the org, and there is no hosted -# fallback. They come back when a box for them does. +# not run on lambda's 22.04. Windows is gone: no self-hosted Windows runner is +# registered and there is no hosted fallback. macOS came back with a box for it +# (#3204): aarch64-apple-darwin builds host-native on mini-m4 (build-darwin). name: Nightly @@ -123,9 +123,62 @@ jobs: apr-${{ matrix.target }}.tar.gz apr-${{ matrix.target }}.tar.gz.sha256 + # ── Build apr: macOS aarch64 natively on mini-m4 (#3204) ── + # Host-native: there is no docker on macOS and no glibc floor to pin, so the + # toolchain is rust-toolchain.toml via the runner's rustup, as in ci.yml's + # mac-check. A LITERAL runs-on list, not fromJSON, so check_runner_labels.sh + # sees the apple-silicon label. One box with 16 GB: this leg is not allowed to + # hold the Linux nightly hostage -- `release` publishes when `build` is green + # whatever this job did, and the darwin tarball is simply absent that night. + build-darwin: + needs: check-activity + if: needs.check-activity.outputs.has_commits == 'true' + name: apr aarch64-apple-darwin on mini + runs-on: [self-hosted, macOS, ARM64, apple-silicon, m4, mini] + timeout-minutes: 90 + env: + CARGO_INCREMENTAL: 0 + CARGO_TERM_COLOR: never + T: aarch64-apple-darwin + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Build apr (host-native, pinned toolchain) + run: | + set -euo pipefail + rustup show active-toolchain + cargo build --release --locked -p apr-cli --target "$T" + "target/$T/release/apr" --version + + - name: Package + run: | + set -euo pipefail + ARCHIVE="apr-$T" + rm -rf "$ARCHIVE" "$ARCHIVE.tar.gz" "$ARCHIVE.tar.gz.sha256" + mkdir -p "$ARCHIVE" + cp "target/$T/release/apr" "$ARCHIVE/" + for f in README.md LICENSE LICENSE-MIT COPYING UNLICENSE; do + if [ -f "$f" ]; then cp "$f" "$ARCHIVE/"; fi + done + tar czf "${ARCHIVE}.tar.gz" "$ARCHIVE" + shasum -a 256 "${ARCHIVE}.tar.gz" > "${ARCHIVE}.tar.gz.sha256" + echo "Packaged: ${ARCHIVE}.tar.gz ($(du -h "${ARCHIVE}.tar.gz" | cut -f1))" + + - name: Upload artifact + uses: actions/upload-artifact@v7 + with: + name: apr-aarch64-apple-darwin + path: | + apr-aarch64-apple-darwin.tar.gz + apr-aarch64-apple-darwin.tar.gz.sha256 + # ── Create/update nightly prerelease ───────────────────── + # Gated on the Linux matrix alone: build-darwin is in `needs` only so its + # artifact is downloaded when it exists (see build-darwin's comment). release: - needs: build + needs: [build, build-darwin] + if: ${{ !cancelled() && needs.build.result == 'success' }} runs-on: [self-hosted, Linux, X64, clean-room] steps: - name: Checkout diff --git a/.github/workflows/rc-cut.yml b/.github/workflows/rc-cut.yml new file mode 100644 index 0000000000..ea86e867c4 --- /dev/null +++ b/.github/workflows/rc-cut.yml @@ -0,0 +1,89 @@ +name: RC Cut + +# Cut vX.Y.Z-rc.N as a GitHub PRERELEASE when CI goes green on the tip of +# release/X.Y.Z (#4285, RC-DOGFOOD-001 §4.1). The decision and every write live in +# scripts/release/rc_cut.sh; this file only wires the trigger and the token. +# +# TRIGGER. ci.yml does not run on a push to release/**. A release branch is gated by +# CI on its release PR (release/X.Y.Z -> main), which re-runs on every push with +# head_sha = the branch tip. `workflow_run` fires when any CI run completes. The +# `branches` filter matches the run's HEAD branch, so a release PR's runs arrive +# here and a feature PR's do not. rc_cut.sh then requires, on that run: +# `ci / gate` and `workspace-test` success, head repository = this repository +# (fork PRs can name a branch release/X.Y.Z), head_sha still the tip, and no rc +# tag already on that commit. Anything else is a logged skip, not a failure. +# +# `workflow_run` always runs this file as it is on the DEFAULT branch, with this +# repository's token, so a release branch cannot change what the cut does. +# +# ASSETS. A release created with GITHUB_TOKEN does not fire `release: published` +# in binary-release.yml (GitHub suppresses recursive triggers), so rc_cut.sh +# dispatches binary-release.yml with inputs.tag. That needs `actions: write`. +# +# PUBLISH GATE (#4287). Before anything is written, the job decides with --dry-run, +# checks the rc commit out beside the default-branch checkout (in RUNNER_TEMP, so no +# cargo config is inherited from this tree) and runs scripts/release/rc_publish_gate.sh +# on it: the publish graph (R6), the path-only dev-dep rule that #3425 broke v0.68.0 +# on, and a `cargo package --no-verify` walk of every publishable crate. Red means the +# job fails and no rc exists; the defect is found on the rc, never at the final tag. +# The real cut then re-decides from the same run, so it tags the commit that was gated. +# +# CI cuts the tag and the prerelease. Publishing a crate stays out of CI (RP-001). + +on: + workflow_run: + workflows: [CI] + types: [completed] + branches: ['release/**'] + +permissions: + contents: write # create the tag ref and the prerelease + actions: write # dispatch binary-release.yml for the assets + +# One cut at a time per release branch, never cancelled: N is read then written, +# and two green runs racing would both read the same N. The tag ref is also the +# lock (a duplicate gets HTTP 422 and writes nothing else). +concurrency: + group: rc-cut-${{ github.event.workflow_run.head_branch }} + cancel-in-progress: false + +jobs: + cut: + # Operator rule (2026-09-10): self-hosted fleet only. The job needs bash, curl + # and python3, which clean-room carries (binary-release.yml's verify job). + runs-on: [self-hosted, Linux, clean-room] + # the package walk is ~65 s warm on lambda; a cold registry index is the rest + timeout-minutes: 30 + # Cheap pre-filter only; rc_cut.sh re-judges every condition itself. + if: github.event.workflow_run.conclusion != 'cancelled' + steps: + - uses: actions/checkout@v7 + with: + # the default branch's script, never the release branch's + ref: ${{ github.event.repository.default_branch }} + fetch-depth: 1 + - name: Case tables (the decision and the gate must still be able to refuse) + run: | + bash scripts/release/rc_cut.sh --self-test + bash scripts/release/rc_publish_gate.sh --self-test + - name: Decide (writes nothing) + id: decide + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + RUN_ID: ${{ github.event.workflow_run.id }} + run: bash scripts/release/rc_cut.sh --run-id "$RUN_ID" --dry-run + - name: Publish gate on the rc commit (#4287) + if: steps.decide.outputs.tag != '' + env: + SHA: ${{ steps.decide.outputs.sha }} + run: | + git fetch --no-tags --depth 1 origin "$SHA" + git worktree add --detach "$RUNNER_TEMP/rc-tree" "$SHA" + bash scripts/release/rc_publish_gate.sh "$RUNNER_TEMP/rc-tree" + - name: Cut the rc + if: success() && steps.decide.outputs.tag != '' # explicit: a red publish gate never reaches the cut + id: cut + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + RUN_ID: ${{ github.event.workflow_run.id }} + run: bash scripts/release/rc_cut.sh --run-id "$RUN_ID" diff --git a/.pmat-gates.toml b/.pmat-gates.toml index 5334a80911..028b330210 100644 --- a/.pmat-gates.toml +++ b/.pmat-gates.toml @@ -100,7 +100,10 @@ exclude = [ # the count did not. Between those readings #3004 added one B- definition # (registry/wgpu_probe.rs entry, 603) and merged, because no required job runs # `pmat comply`; it was split into four graded tables in the pin-bump PR, back to 602. -baseline = 599 +baseline = 598 +# 598 re-measured 2026-09-25 under pmat 3.41.1 from a COLD index (XDG_CACHE_HOME in a fresh temp dir) +# on batch/0.70.0 + #4101: 603 there before #4101 split token_usage_report.main, shapes.parse_property, +# shapes.well_formed, crux_inference_judge.collect and model_ladder_cells.judge. Lowered, never raised. # 599 re-measured 2026-09-18 under pmat 3.41.1 (the fleet pin, infra#692) from a cold index at # 3409b29d2. Instrument vs tree, separated: 3.41.1 reads 602 on 5d95ed54e, the tree that banked # 601 under 3.40.2 (+1 from the instrument on that tree, definition not identified), and 599 diff --git a/CLAUDE.md b/CLAUDE.md index 6cc9c0e5a3..a8c0e00ef3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -108,7 +108,7 @@ make tier1 # Fast feedback (<1s): fmt, clippy, check make tier2 # Pre-commit (<5s): tests + strict clippy make tier3 # Pre-push (1-5min): full validation + coverage make tier4 # CI/CD: includes pmat analysis -make coverage # Coverage report (enforced floor 88%, target ≥95%) +make coverage # Coverage report (enforced floor: COV_FLOOR in Makefile, target ≥95%) ``` ## Debugging: Use apr Tools First (MANDATORY) @@ -404,7 +404,7 @@ ran (#2361). When a fix seems to have no effect, ask what else claims that name ## Testing -Target: 60% unit, 30% property, 10% integration. Coverage: **88.78% line** (786448/885829, measured 2026-07-29 by coverage-nightly on 95145584f; target ≥95%, enforced floor 88% via COV_FLOOR). The long-quoted "96.35%" predates the measurement ever working - the pipeline reported 0/0 until #2333. +Target: 60% unit, 30% property, 10% integration. Coverage: **90.84% line** (849706/935347, measured 2026-09-24 by a manual `workflow_dispatch` of coverage-nightly (run 36065904363) on branch `PMAT-3839-cov-regex-scope` @ 237f10f8b, folded into batch/0.70.0 and NOT on `main` until that batch merges; `main`'s own nightly (49fe19c28, 2026-09-24) still measured 826969/941649 = 87.82% under the old regex (run 35937679357); target ≥95%, enforced floor 89% via COV_FLOOR). Before #3839 the exclude regex still described the pre-monorepo tree, so the 88.78% of 2026-07-29 measured a different set of files. The long-quoted "96.35%" predates the measurement ever working - the pipeline reported 0/0 until #2333. ```bash cargo test -p --lib # Unit tests for one crate (what you run while working) @@ -508,7 +508,7 @@ still carries a header comment claiming it was "Updated for PMAT v2.215.0". | Score | Value | Provenance | |-------|-------|------------| -| Line coverage | **88.78%** (786448/885829) | coverage-nightly, 2026-07-29, commit `95145584f`. The long-quoted "96.35%"/"96.94%" predates the pipeline ever working — it reported 0/0 until #2333 | +| Line coverage | **90.84%** (849706/935347) | coverage-nightly `workflow_dispatch` run 36065904363, 2026-09-24, on branch commit `237f10f8b` (monorepo scope, #3839; in batch/0.70.0, not yet `main`, whose scheduled nightly 35937679357 read 87.82%). The previous 88.78% (2026-07-29, `95145584f`) used the pre-monorepo exclude regex. The long-quoted "96.35%"/"96.94%" predates the pipeline ever working — it reported 0/0 until #2333 | | Project score / TDG / mutation % | **re-derive** — `pmat rust-project-score`, `pmat tdg . --include-components`, `cargo mutants` | The previously published "124/134", "TDG 95.2/100" and "Mutation 85.3%" carried no date or commit and could not be reproduced from the tree | **Thresholds — read from the config, which does not say what this file used to say:** @@ -516,7 +516,7 @@ still carries a header comment claiming it was "Updated for PMAT v2.215.0". | Gate | Configured as | Where | |------|---------------|-------| | Coverage (aspirational) | `min_coverage = 95.0` | `.pmat-gates.toml` | -| Coverage (**enforced**) | `COV_FLOOR := 88` — the last *measured* value, and the one that actually fails a build | `Makefile:287`, `.github/workflows/coverage-nightly.yml` | +| Coverage (**enforced**) | `COV_FLOOR := 89` — a ratchet under the last *measured* value, and the one that actually fails a build | `Makefile` (`COV_FLOOR`), `.github/workflows/coverage-nightly.yml` | | Cyclomatic complexity | `max_complexity = 10` per fn | `.pmat-gates.toml` | | TDG | `min_grade = "B"` — **not** "≥95" | `.pmat-gates.toml` `[tdg]` | | Mutation | `MUTANTS_MAX_MISSED` (default **0**) surviving mutants **on the PR diff** — not a global 85% score | `.github/workflows/ci.yml:517` | diff --git a/Cargo.lock b/Cargo.lock index 7f4bdeb53d..2a207bd6c7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -251,6 +251,7 @@ name = "apr-cli" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "aprender-cgp", "aprender-common", "aprender-compute", @@ -265,6 +266,7 @@ dependencies = [ "aprender-present-core", "aprender-present-terminal", "aprender-profile", + "aprender-ptx-debug", "aprender-quant", "aprender-rag-cli", "aprender-registry", @@ -364,11 +366,19 @@ dependencies = [ "tokenizers 0.22.2", ] +[[package]] +name = "aprender-build-sha" +version = "0.69.0" +dependencies = [ + "tempfile", +] + [[package]] name = "aprender-cbtop" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "aprender-common", "aprender-compute", "aprender-gpu", @@ -394,6 +404,7 @@ name = "aprender-cgp" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "aprender-gpu", "chrono", "clap", @@ -413,6 +424,7 @@ dependencies = [ name = "aprender-common" version = "0.69.0" dependencies = [ + "clap", "lz4_flex 0.11.6", "zstd", ] @@ -422,6 +434,7 @@ name = "aprender-compute" version = "0.69.0" dependencies = [ "anyhow", + "aprender-contracts", "aprender-contracts-macros", "aprender-core", "aprender-cuda-edge", @@ -471,6 +484,7 @@ name = "aprender-compute-xtask" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "bashrs", "clap", "colored 2.2.0", @@ -502,6 +516,8 @@ dependencies = [ name = "aprender-contracts-cli" version = "0.69.0" dependencies = [ + "aprender-build-sha", + "aprender-common", "aprender-contracts", "clap", "serde", @@ -616,6 +632,7 @@ name = "aprender-data" version = "0.69.0" dependencies = [ "aes-gcm", + "aprender-build-sha", "aprender-test-lib", "argon2", "arrow 59.2.0", @@ -668,8 +685,10 @@ name = "aprender-db" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "aprender-common", "aprender-compute", + "aprender-contracts", "arrow 59.2.0", "axum 0.7.9", "bytemuck", @@ -742,6 +761,7 @@ dependencies = [ name = "aprender-explain" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-gpu", "aprender-present-core", "aprender-present-terminal", @@ -860,6 +880,7 @@ name = "aprender-orchestrate" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", "aprender-common", "aprender-compute", "aprender-contracts", @@ -929,6 +950,8 @@ dependencies = [ name = "aprender-present-cli" version = "0.69.0" dependencies = [ + "aprender-build-sha", + "aprender-contracts", "aprender-present-yaml", "clap", "notify", @@ -942,6 +965,7 @@ dependencies = [ name = "aprender-present-core" version = "0.69.0" dependencies = [ + "aprender-contracts", "aprender-contracts-macros", "criterion 0.7.0", "proptest", @@ -994,6 +1018,7 @@ dependencies = [ name = "aprender-present-terminal" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-present-core", "bitvec", "clap", @@ -1058,6 +1083,7 @@ version = "0.69.0" dependencies = [ "addr2line 0.25.1", "anyhow", + "aprender-build-sha", "aprender-compute", "aprender-core", "aprender-db", @@ -1118,6 +1144,7 @@ dependencies = [ name = "aprender-ptx-debug" version = "0.69.0" dependencies = [ + "aprender-build-sha", "clap", "proptest", "thiserror 2.0.18", @@ -1127,6 +1154,7 @@ dependencies = [ name = "aprender-qa-certify" version = "0.69.0" dependencies = [ + "aprender-build-sha", "chrono", "clap", "proptest", @@ -1138,6 +1166,7 @@ dependencies = [ name = "aprender-qa-cli" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-qa-certify", "aprender-qa-gen", "aprender-qa-report", @@ -1227,6 +1256,7 @@ version = "0.69.0" dependencies = [ "aprender-common", "aprender-compute", + "aprender-contracts", "aprender-serve", "async-trait", "bincode", @@ -1254,6 +1284,8 @@ name = "aprender-rag-cli" version = "0.69.0" dependencies = [ "anyhow", + "aprender-build-sha", + "aprender-common", "aprender-rag", "assert_cmd", "blake3", @@ -1396,6 +1428,7 @@ dependencies = [ name = "aprender-simulate" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-contracts", "aprender-contracts-macros", "aprender-present-core", @@ -1464,6 +1497,7 @@ dependencies = [ name = "aprender-test-cli" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-test-lib", "assert_cmd", "atty", @@ -1650,6 +1684,7 @@ dependencies = [ name = "aprender-train-bench" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-train", "aprender-train-common", "assert_cmd", @@ -1679,6 +1714,7 @@ dependencies = [ name = "aprender-train-distill" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-train", "aprender-train-common", "assert_cmd", @@ -1700,6 +1736,7 @@ dependencies = [ name = "aprender-train-inspect" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-train", "aprender-train-common", "assert_cmd", @@ -1716,6 +1753,7 @@ dependencies = [ name = "aprender-train-lora" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-train", "aprender-train-common", "assert_cmd", @@ -1732,6 +1770,7 @@ dependencies = [ name = "aprender-train-shell" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-train", "aprender-train-common", "clap", @@ -1786,6 +1825,7 @@ dependencies = [ name = "aprender-verify-ml" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-compute", "aprender-core", "aprender-profile", @@ -1863,6 +1903,7 @@ dependencies = [ name = "aprender-zram-cli" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-zram-adaptive", "aprender-zram-core", "clap", @@ -1889,6 +1930,7 @@ dependencies = [ name = "aprender-zram-generator" version = "0.69.0" dependencies = [ + "aprender-build-sha", "aprender-zram-core", "clap", "serde", @@ -3787,9 +3829,9 @@ dependencies = [ [[package]] name = "chacha20" -version = "0.10.0" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601" +checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06" dependencies = [ "cfg-if 1.0.4", "cpufeatures 0.3.0", @@ -4990,8 +5032,18 @@ version = "0.20.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fc7f46116c46ff9ab3eb1597a45688b6715c6e628b5c133e288e709a29bcb4ee" dependencies = [ - "darling_core", - "darling_macro", + "darling_core 0.20.11", + "darling_macro 0.20.11", +] + +[[package]] +name = "darling" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "25ae13da2f202d56bd7f91c25fba009e7717a1e4a1cc98a76d844b65ae912e9d" +dependencies = [ + "darling_core 0.23.0", + "darling_macro 0.23.0", ] [[package]] @@ -5008,13 +5060,37 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "darling_core" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9865a50f7c335f53564bb694ef660825eb8610e0a53d3e11bf1b0d3df31e03b0" +dependencies = [ + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 2.0.118", +] + [[package]] name = "darling_macro" version = "0.20.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fc34b93ccb385b40dc71c6fceac4b2ad23662c7eeb248cf10d529b7e055b6ead" dependencies = [ - "darling_core", + "darling_core 0.20.11", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "darling_macro" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3984ec7bd6cfa798e62b4a642426a5be0e68f9401cfc2a01e3fa9ea2fcdb8d" +dependencies = [ + "darling_core 0.23.0", "quote", "syn 2.0.118", ] @@ -5131,7 +5207,7 @@ version = "0.20.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2d5bcf7b024d6835cfb3d473887cd966994907effbe9227e8c8219824d06c4e8" dependencies = [ - "darling", + "darling 0.20.11", "proc-macro2", "quote", "syn 2.0.118", @@ -9415,13 +9491,13 @@ dependencies = [ [[package]] name = "papergrid" -version = "0.12.0" +version = "0.18.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c7419ad52a7de9b60d33e11085a0fe3df1fbd5926aa3f93d3dd53afbc9e86725" +checksum = "d0984e668274d34691bc2b262ef0d115de5fa9973bcdee7ae32213f93099153e" dependencies = [ "bytecount", "fnv", - "unicode-width 0.1.11", + "unicode-width 0.2.0", ] [[package]] @@ -9982,49 +10058,25 @@ dependencies = [ ] [[package]] -name = "proc-macro-error" -version = "1.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da25490ff9892aab3fcf7c36f08cfb902dd3e71ca0f9f9517bea02a73a5ce38c" -dependencies = [ - "proc-macro-error-attr", - "proc-macro2", - "quote", - "syn 1.0.109", - "version_check", -] - -[[package]] -name = "proc-macro-error-attr" -version = "1.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a1be40180e52ecc98ad80b184934baf3d0d29f979574e439af5a55274b35f869" -dependencies = [ - "proc-macro2", - "quote", - "version_check", -] - -[[package]] -name = "proc-macro-error-attr2" -version = "2.0.0" +name = "proc-macro-error-attr3" +version = "3.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "96de42df36bb9bba5542fe9f1a054b8cc87e172759a1868aa05c1f3acc89dfc5" +checksum = "9e564d14133360e1ae169ffde5da25881b5fa47261665b8e5713c212c27799da" dependencies = [ "proc-macro2", "quote", ] [[package]] -name = "proc-macro-error2" -version = "2.0.1" +name = "proc-macro-error3" +version = "3.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "11ec05c52be0a07b08061f7dd003e7d7092e0472bc731b4af7bb1ef876109802" +checksum = "8f0d4471b3436c22106b21913b1dda531558918ae9b7ec55d58aa84b43552233" dependencies = [ - "proc-macro-error-attr2", + "proc-macro-error-attr3", "proc-macro2", "quote", - "syn 2.0.118", + "syn 3.0.6", ] [[package]] @@ -10398,7 +10450,7 @@ version = "0.10.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d2e8e8bcc7961af1fdac401278c6a831614941f6164ee3bf4ce61b7edb162207" dependencies = [ - "chacha20 0.10.0", + "chacha20 0.10.2", "getrandom 0.4.3", "rand_core 0.10.1", ] @@ -12262,6 +12314,16 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "syn" +version = "3.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee" +dependencies = [ + "proc-macro2", + "unicode-ident", +] + [[package]] name = "sync_wrapper" version = "1.0.2" @@ -12408,25 +12470,25 @@ dependencies = [ [[package]] name = "tabled" -version = "0.16.0" +version = "0.22.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "77c9303ee60b9bedf722012ea29ae3711ba13a67c9b9ae28993838b63057cb1b" +checksum = "2d2596a104db1900b943f97d793a6c99addca918c4525c999f751a0f17d472eb" dependencies = [ "papergrid", "tabled_derive", + "testing_table", ] [[package]] name = "tabled_derive" -version = "0.8.0" +version = "0.12.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bf0fb8bfdc709786c154e24a66777493fb63ae97e3036d914c8666774c477069" +checksum = "7dca1937322a1e892b1a65f6a6736183bb0a29d3b4234d1d53bc436dff9beb76" dependencies = [ - "heck 0.4.1", - "proc-macro-error", + "heck 0.5.0", "proc-macro2", "quote", - "syn 1.0.109", + "syn 2.0.118", ] [[package]] @@ -12496,6 +12558,15 @@ version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f50febec83f5ee1df3015341d8bd429f2d1cc62bcba7ea2076759d315084683" +[[package]] +name = "testing_table" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0f8daae29995a24f65619e19d8d31dea5b389f3d853d8bf297bbf607cd0014cc" +dependencies = [ + "unicode-width 0.2.0", +] + [[package]] name = "thiserror" version = "1.0.69" @@ -13774,12 +13845,11 @@ dependencies = [ [[package]] name = "validator" -version = "0.20.0" +version = "0.21.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "43fb22e1a008ece370ce08a3e9e4447a910e92621bb49b85d6e48a45397e7cfa" +checksum = "c3d68c6633c483df6780cc5277a417c7c2d1bceee2649d06c8ab6b0fd2dd3c81" dependencies = [ "idna", - "once_cell", "regex", "serde", "serde_derive", @@ -13790,13 +13860,12 @@ dependencies = [ [[package]] name = "validator_derive" -version = "0.20.0" +version = "0.20.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b7df16e474ef958526d1205f6dda359fdfab79d9aa6d54bafcb92dcd07673dca" +checksum = "240e4b81c20a1d6d50d1d7265c658dfbd204e8b9ac4d80f3c931f39462196335" dependencies = [ - "darling", - "once_cell", - "proc-macro-error2", + "darling 0.23.0", + "proc-macro-error3", "proc-macro2", "quote", "syn 2.0.118", @@ -15594,7 +15663,7 @@ version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a76ff259533532054cfbaefb115c613203c73707017459206380f03b3b3f266e" dependencies = [ - "darling", + "darling 0.20.11", "proc-macro2", "quote", "syn 2.0.118", diff --git a/Cargo.toml b/Cargo.toml index 319898b3a9..4e4b35c622 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,6 +4,8 @@ members = [ "crates/aprender-core", # --- APR-2231: sovereign `.apr` container leaf crate --- "crates/apr-format", + # --- #4219: shared build.rs helper that stamps APR_GIT_SHA into every [[bin]] --- + "crates/aprender-build-sha", # --- Pre-existing aprender crates --- "crates/aprender-shell", "crates/aprender-tsp", @@ -294,11 +296,13 @@ aprender-train-common = { path = "crates/aprender-train-common", version = "0.69 aprender-train-distill = { path = "crates/aprender-train-distill", version = "0.69.0" } aprender-orchestrate = { path = "crates/aprender-orchestrate", version = "0.69.0" } aprender-contracts = { path = "crates/aprender-contracts", version = "0.69.0" } +aprender-build-sha = { path = "crates/aprender-build-sha", version = "0.69.0" } aprender-profile = { path = "crates/aprender-profile", version = "0.69.0" } aprender-profile-core = { path = "crates/aprender-profile-core", version = "0.69.0" } aprender-zram-core = { path = "crates/aprender-zram-core", version = "0.69.0" } aprender-zram-adaptive = { path = "crates/aprender-zram-adaptive", version = "0.69.0" } aprender-zram-cli = { path = "crates/aprender-zram-cli", version = "0.69.0" } +aprender-ptx-debug = { path = "crates/aprender-ptx-debug", version = "0.69.0" } aprender-present-core = { path = "crates/aprender-present-core", version = "0.69.0" } aprender-present-terminal = { path = "crates/aprender-present-terminal", version = "0.69.0" } aprender-present-widgets = { path = "crates/aprender-present-widgets", version = "0.69.0" } @@ -642,6 +646,16 @@ gates = [ # Release time only, like the entry above; the PR-time half (the helpers' case table) is # scripts/check_tokenizer_parity.sh, which guard_tree.sh dispatches. "scripts/tokenizer_parity.sh", + # #3740 — in-tree tests that compare against llama.cpp read ONLY the pinned build that + # scripts/llama_bin.sh proves and exports; without it they skip. The release runs them WITH it, + # and a skip here is a FAIL, so a skip is never the release path. + "scripts/dogfood_comparator_env_tests.sh", + # #4114: every publishable crate's tests compile FROM ITS .crate (packaged, unpacked outside + # the tree, siblings patched to their tarballs). The source reader check_package_includes.sh + # skipped #[cfg(test)] modules and passed aprender-serve 0.69.1, whose tarball cannot compile + # its tests. Release time only: one workspace test build. The PR-time half is its case table, + # rows 14-16 of check_package_includes.sh --self-test (ci.yml guard-cargo). + "scripts/package_tarball_build.sh", ] # The interfaces this release SHIPS, each paired with the e2e target that proves diff --git a/Makefile b/Makefile index 064caf861a..4c333aff9b 100644 --- a/Makefile +++ b/Makefile @@ -456,9 +456,18 @@ tier4: tier3 # ============================================================================ # COVERAGE TARGETS (Two-Phase Pattern from bashrs) # ============================================================================ -# Pattern: bashrs/Makefile - Two-phase coverage with mold linker workaround -# CRITICAL: mold linker breaks LLVM coverage instrumentation -# Solution: Temporarily move ~/.cargo/config.toml during coverage runs +# #3839: these targets used to `mv ~/.cargo/config.toml` aside for the whole run +# (the bashrs "mold breaks LLVM coverage" workaround). That renames a file every +# other cargo process of this user reads, so on a shared host it silently changes +# other agents' builds mid-run. No fleet host has a global config (checked +# 2026-09-23: lambda, intel, yoga, gx10), so the move was a no-op there. Instead, +# COV_REFUSE_GLOBAL_MOLD refuses to measure where a global config enables mold, +# and never edits it. +COV_REFUSE_GLOBAL_MOLD = @if [ -f "$${CARGO_HOME:-$$HOME/.cargo}/config.toml" ] && grep -q mold "$${CARGO_HOME:-$$HOME/.cargo}/config.toml"; then \ + echo "❌ $${CARGO_HOME:-$$HOME/.cargo}/config.toml enables mold, which breaks LLVM coverage instrumentation."; \ + echo " Refusing rather than moving a file every other cargo process on this host reads (issue 3839)."; \ + echo " Run with CARGO_HOME pointing at a copy without mold, or remove mold from that file."; \ + exit 1; fi # Exclusion patterns for coverage reports # ONLY excludes truly external/feature-gated code - all apr subcommands INCLUDED @@ -476,10 +485,14 @@ tier4: tier3 # Test infrastructure: # - test_factory : Test code, not production # - demo/ : Demo/example code -# NOTE: Coverage tracks the main aprender library only. -# Subcrate tests still RUN (--workspace), exercising main lib code paths, -# but subcrate source files are excluded from the coverage REPORT. -# External deps (trueno, realizar, .cargo) also excluded. +# NOTE (#3839): coverage measures the MONOREPO. #4023 scopes every report by a derived +# `-p` list, so aprender-serve/-train/-compute (formerly realizar/entrenar/trueno) are +# measured. The pre-monorepo `trueno|realizar/|entrenar/` alternatives were removed: in-tree +# `entrenar/` matched 0 files, `realizar/` 7 unrelated aprender-train files, and `trueno` +# 54/56 of aprender-zram while missing aprender-compute entirely. +# aprender-compute/src/backends/gpu/ : no coverage runner executes it (no GPU lane), so it +# is kept out of the denominator until one exists - excluded AND unrun, never "measured 0%". +# Named subcrates below (apr-cli, aprender-shell, ...) and .cargo stay excluded. # Subcrate code, external deps, and modules requiring external model files for coverage. # models/ = dead code per UCBD §9.1 (scheduled for deletion). # serialization/ = SafeTensors IO (needs actual .safetensors files). @@ -489,7 +502,7 @@ tier4: tier3 # format/rosetta = cross-format parity (needs model files). # transfer/ = transfer learning (needs pretrained models). # bench/ = benchmark visualization (non-core). -COVERAGE_EXCLUDE_REGEX := \.cargo/|trueno|realizar/|entrenar/|fuzz/|golden_traces/|hf_hub/|demo/|test_factory|pacha/|showcase/|apr-cli/|aprender-shell/|aprender-tsp/|aprender-monte-carlo/|chaos\.rs|audio/|format/quantize\.rs|format/signing\.rs|voice/|playback\.rs|rustlib/src/rust|models/|serialization/|speech/|format/onnx|format/converter|format/rosetta|transfer/|bench_viz/ +COVERAGE_EXCLUDE_REGEX := \.cargo/|aprender-compute/src/backends/gpu/|fuzz/|golden_traces/|hf_hub/|demo/|test_factory|pacha/|showcase/|apr-cli/|aprender-shell/|aprender-tsp/|aprender-monte-carlo/|chaos\.rs|audio/|format/quantize\.rs|format/signing\.rs|voice/|playback\.rs|rustlib/src/rust|models/|serialization/|speech/|format/onnx|format/converter|format/rosetta|transfer/|bench_viz/ # Coverage threshold (enforced: fail if below) COV_THRESHOLD := 95 @@ -505,7 +518,13 @@ COV_THRESHOLD := 95 # So the enforced condition is "do not regress below what we actually have". # Raise this number whenever a run comes in higher; never lower it to make red # go away. Integer truncation gives ~0.78pt of headroom before 88 becomes 87. -COV_FLOOR := 88 +# 2026-09-23, #4023: 88 -> 89. The first COMPLETE measurement (every aprender-serve process +# exited normally; coverage-nightly run 35908686532) was 849871/941605 = 90.26%. 89 is a +# ratchet with margin, since 90 would leave no room for noise; it goes to 90 once two +# consecutive nightlies measure >= 90.5% (release-cop ruling). +COV_FLOOR := 89 +# #4023: libtest threads for aprender-serve's `gpu` coverage shard (25.9 GB at 22 on yoga). +COV_GPU_SHARD_THREADS ?= 4 # NVMe target dir (mirrors cargo() shell function that sets CARGO_TARGET_DIR) # Without this, Make's subshell bypasses the function and uses ./target/ instead @@ -535,10 +554,13 @@ COV_CARGO_ENV := $(if $(COV_TARGET_DIR),CARGO_TARGET_DIR=$(COV_TARGET_DIR)) # two-phase, unscoped report -> LH=0 LF=0 (empty) # report --summary-only -p A -p B -> LH=686 LF=737 (93.08%) # single-phase --lcov --output-path -> LH=686 LF=737 (93.08%) -# Single-phase is chosen over an explicit -p list because the invocation that selects the -# scope is the one that writes the report, so the two cannot drift apart again. profraw +# #4023 brings two-phase BACK, deliberately: aprender-serve's lib tests cannot run in one +# process on a 28 GB runner (#4028), so they run as several --no-report processes and one +# report merges them. It is safe because every report is now scoped by an explicit `-p` list +# DERIVED from `cargo metadata` (scripts/coverage_report_scope.py), the verified alternative +# above, and scripts/check_coverage_report_scoped.sh refuses any unscoped `llvm-cov report`. profraw # survive it (31 present afterwards), so coverage-html still has data to work from. -.PHONY: coverage-check contracts +.PHONY: coverage-check contracts census # BSE-03 phase A (Pmat-Ticket: PMAT-1068). The README's contract count is # DERIVED: scripts/readme_sync.sh rewrites the text between the @@ -583,16 +605,36 @@ coverage-check: coverage # CLAUDE.md, and the dogfood protocol looked for a target that did not exist, so # it WARNed instead of checking. `pv lint` runs validate + audit + score across # contracts/ and is the documented entry point (never hand-rolled bash). +# The release train's one census writer (#3569). A PR that runs this and commits +# the result is refused by scripts/check_census_derived.sh; the train runs it on +# release/X.Y.Z, where the guard exempts the edit. +census: + @. scripts/pv_bin.sh && t=$$(mktemp contracts/census.json.XXXXXX) && "$$PV" census contracts --format json > "$$t" && bash scripts/check_census_derived.sh --census "$$t" && mv "$$t" contracts/census.json || { rm -f "$$t"; exit 1; } + @bash scripts/readme_sync.sh --write + +# EXIT PROPAGATION (PVL-001 EV-4, aprender#4168). Under .ONESHELL this whole +# recipe is ONE shell script, so without errexit its status is the LAST line's +# and every earlier step -- `pv lint` included -- was advisory: a failing lint +# printed its tail and the gate exited 0. `set -e` stops at a failing step. +# It is NOT enough on the pv lines: errexit ignores a failure on the LEFT of +# `&&`, so a pv_bin.sh that REFUSES the binary (stale, wrong identity) would +# fall through to the next step. Hence `|| exit` there as well, which exits +# with that list's own status (pv's rc through the pipe, via -o pipefail). +# Case table + mutants: scripts/tests/make_contracts_propagates.sh. contracts: + @set -e @echo "== provable contracts: pv lint contracts/ ==" - @. scripts/pv_bin.sh && "$$PV" lint contracts/ 2>&1 | tail -5 - @echo "== census: tracked contracts/census.json == a fresh one (ONT-001 ONT-1, F-1) ==" - @git ls-files --error-unmatch contracts/census.json >/dev/null || { echo "FAIL: contracts/census.json is not tracked, so diffing it proves nothing"; exit 1; } - @. scripts/pv_bin.sh && "$$PV" census contracts --format json > contracts/census.json - @git diff --exit-code contracts/census.json || { echo "FAIL: the tracked census differs from a fresh one — commit the regenerated contracts/census.json"; exit 1; } + @. scripts/pv_bin.sh && "$$PV" lint contracts/ 2>&1 | tail -5 || exit + @echo "== census: a FRESH pv census holds its invariants (ONT-001 ONT-1, F-1; #3569) ==" + @# The tracked contracts/census.json is a release-train snapshot (its one writer, + @# `make census`) and is expected to lag; a PR may not edit it + @# (scripts/check_census_derived.sh). So the census is computed fresh into a temp + @# file, its identities are checked there, and the README count (readme_sync's + @# tree listing) must equal its n_files — the tracked file is not rewritten. + @. scripts/pv_bin.sh && t=$$(mktemp) && ( "$$PV" census contracts --format json > "$$t" && bash scripts/check_census_derived.sh --census "$$t" && CENSUS_JSON="$$t" bash scripts/readme_sync.sh --check; rc=$$?; rm -f "$$t"; exit $$rc ) || exit @echo "== graph: tracked contracts/contracts.nt + shapes.ttl == a fresh extraction (ONT-001 ONT-4b, R-18) ==" - @. scripts/pv_bin.sh && "$$PV" extract contracts --check >/dev/null - @echo "== README states the censused count ==" + @. scripts/pv_bin.sh && "$$PV" extract contracts --check >/dev/null || exit + @echo "== README states the tree's count (the listing readme_sync uses; the fresh census was held to it above) ==" @bash scripts/readme_sync.sh --check @echo "== provenance marks, interim (ONT-001 R-10) ==" @bash scripts/lint-provenance.sh --self-test @@ -600,38 +642,121 @@ contracts: @echo "== contract engine tests ==" @cargo test -p aprender-contracts --lib 2>&1 | grep -E "test result" | tail -1 +# #3839: skips are EXACT full test paths from scripts/coverage-skips.txt, one reason +# per entry. They used to be 19 --skip substrings that removed 2,713 tests (2,702 of +# which pass without a GPU), so the number measured a subset over the whole denominator. coverage: ## Coverage summary + threshold check (warm: ~3min) @echo "📊 Running coverage ($(COV_THRESHOLD)%+ threshold)..." + @# #4023: refuse before any test runs if a `llvm-cov report` anywhere would cover only the facade. + @scripts/check_coverage_report_scoped.sh @which cargo-llvm-cov > /dev/null 2>&1 || { cargo install cargo-llvm-cov --locked || exit 1; } - @test -f ~/.cargo/config.toml && mv ~/.cargo/config.toml ~/.cargo/config.toml.bak || true + $(COV_REFUSE_GLOBAL_MOLD) @# Pre-clean: remove stale profraw files to avoid LLVM version mismatch @COVDIR=$$($(COV_CARGO_ENV) cargo llvm-cov show-env 2>/dev/null | grep CARGO_LLVM_COV_TARGET_DIR | sed "s/.*=//"); \ if [ -n "$$COVDIR" ]; then find "$$COVDIR" -name '*.profraw' -delete 2>/dev/null || true; fi @mkdir -p target/coverage + @rm -f target/coverage/lcov.info target/coverage/test.log target/coverage/failed-tests.txt @printf '%s' '$(COVERAGE_EXCLUDE_REGEX)' > target/coverage/.exclude-re - @echo "🧪 Tests with instrumentation + report in ONE invocation (CB-127-A: cargo llvm-cov test, not nextest)..." + @# #4023: aprender-serve's lib tests run as SEVERAL processes. In one process they build up + @# memory across tests (#4028: 30 GB single-threaded, 45 GB at 22 threads on gx10) and earlyoom + @# SIGTERMed them on yoga's 28 GB box (run 35868368976); one module group per process peaks + @# <= 7.8 GB. EVERY run is --no-report and ONE `cargo llvm-cov report` merges them: a run WITH a + @# report cleans the earlier profiles (measured: the first run's coverage fell to 0). + @echo "🧪 Workspace lib tests except aprender-serve (instrumented, --no-report)..." @PROPTEST_CASES=10 QUICKCHECK_TESTS=10 RUST_MIN_STACK=16777216 CARGO_BUILD_JOBS=4 \ - $(COV_CARGO_ENV) cargo llvm-cov test \ - --workspace --exclude aprender-gpu --lib \ - --lcov --output-path target/coverage/lcov.info \ - --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" \ - -- --skip prop_gbm_expected_value --skip slow --skip heavy --skip h12_ --skip j2_ \ - --skip falsification --skip chaos --skip disconnect --skip benchmark_parity \ - --skip qwen2_generation --skip qwen2_golden --skip qwen2_weight --skip load_test \ - --skip spec_checklist_w --skip spec_checklist_u --skip verify_audio --skip g9_roofline \ - --skip cuda --skip gpu_ \ - || { test -f ~/.cargo/config.toml.bak && mv ~/.cargo/config.toml.bak ~/.cargo/config.toml; exit 1; } + $(COV_CARGO_ENV) cargo llvm-cov test --no-report \ + --workspace --exclude aprender-gpu --exclude aprender-serve --lib --ignore-run-fail \ + -- --exact $$(sed -e '/^#/d' -e '/^[[:space:]]*$$/d' -e 's/^/--skip /' scripts/coverage-skips.txt) \ + 2>&1 | tee target/coverage/test.log; \ + rc=$${PIPESTATUS[0]}; \ + if [ "$$rc" -ne 0 ]; then \ + echo "❌ coverage DID NOT MEASURE: cargo llvm-cov exited $$rc on the workspace run (build failure;"; \ + echo " with --ignore-run-fail a failing test alone does not stop it). No coverage verdict."; \ + exit 1; \ + fi + @echo "🧪 aprender-serve lib tests, one process per module group (instrumented, --no-report)..." + @rm -rf target/coverage/serve-shards + @$(COV_CARGO_ENV) cargo llvm-cov test --no-report -p aprender-serve --lib -- --list \ + > target/coverage/serve-list.txt 2>> target/coverage/test.log || \ + { echo "❌ coverage DID NOT MEASURE: could not list aprender-serve's lib tests. No coverage verdict."; exit 1; } + @python3 scripts/coverage_serve_shards.py target/coverage/serve-list.txt scripts/coverage-skips.txt \ + target/coverage/serve-shards scripts/coverage-solo.txt + @# scripts/coverage-solo.txt: run FIRST, each in its OWN process, and print its test binary's peak RSS + @# (RUSAGE_CHILDREN.ru_maxrss), so a later skip carries a measured per-test reason. + @: > target/coverage/failed-runs.txt; \ + for solo in target/coverage/serve-shards/solo-*.txt; do \ + [ -e "$$solo" ] || continue; \ + t=$$(cat $$solo); \ + PROPTEST_CASES=10 QUICKCHECK_TESTS=10 RUST_MIN_STACK=16777216 CARGO_BUILD_JOBS=4 \ + $(COV_CARGO_ENV) python3 -c 'import resource, subprocess, sys; rc = subprocess.call(sys.argv[2:]); print("coverage-solo-maxrss", resource.getrusage(resource.RUSAGE_CHILDREN).ru_maxrss, "KB", sys.argv[1], "rc=%d" % rc, flush=True); sys.exit(rc)' \ + "$$t" cargo llvm-cov test --no-report -p aprender-serve --lib --ignore-run-fail -- --exact "$$t" \ + 2>&1 | tee -a target/coverage/test.log; \ + rc=$${PIPESTATUS[0]}; \ + [ "$$rc" -eq 0 ] || echo "solo $$t rc=$$rc" >> target/coverage/failed-runs.txt; \ + done + @# The `gpu` module builds up memory in one process on yoga (25.9 GB at 22 threads, 26.5 GB at 4; + @# runs 35881004821, 35885731831), so the partitioner chunks it into <= 200-test processes, which + @# also run at COV_GPU_SHARD_THREADS. EVERY shard runs even if one fails, so a dispatch yields the + @# whole picture; any failure then means no verdict, naming each failed shard. + @for shard in target/coverage/serve-shards/shard-*.txt; do \ + threads=""; case "$$shard" in *-gpu.*.txt) threads="--test-threads=$(COV_GPU_SHARD_THREADS)" ;; esac; \ + echo " $$shard ($$(wc -l < $$shard) tests) $$threads"; \ + PROPTEST_CASES=10 QUICKCHECK_TESTS=10 RUST_MIN_STACK=16777216 CARGO_BUILD_JOBS=4 \ + $(COV_CARGO_ENV) cargo llvm-cov test --no-report -p aprender-serve --lib --ignore-run-fail \ + -- --exact $$threads $$(cat $$shard) 2>&1 | tee -a target/coverage/test.log; \ + rc=$${PIPESTATUS[0]}; \ + echo " coverage-shard-rc $$rc $$shard"; \ + [ "$$rc" -eq 0 ] || echo "shard $$shard rc=$$rc" >> target/coverage/failed-runs.txt; \ + done + @if [ -s target/coverage/failed-runs.txt ]; then \ + echo "❌ coverage DID NOT MEASURE: these aprender-serve runs failed (every one was still run):"; \ + sed 's/^/ /' target/coverage/failed-runs.txt; \ + echo " No coverage verdict."; \ + exit 1; \ + fi + @echo "📊 Merging every run's profiles into one report..." + @# `--workspace --exclude aprender-gpu` is REQUIRED: the root Cargo.toml is also a package (the + @# `apr` facade), and an unqualified `report` covers ONLY the root package. Proof run + @# 35892421393 printed "Finished report saved" and then found no (non-empty) lcov. Measured with + @# cargo-llvm-cov 0.9.0 (CI's version) on a root-package workspace: without --workspace the + @# lcov held only src/lib.rs; with it, every member. + @# SCOPE IS EXPLICIT: an unscoped `report` covers only the root facade (empty lcov, run + @# 35892421393 and the single-phase note above); `report --exclude` is rejected by 0.9.0 (run + @# 35901458111) and `report --workspace` by older versions. A derived `-p` list works on both. + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py --exclude aprender-gpu) \ + --lcov --output-path $(CURDIR)/target/coverage/lcov.info \ + --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" 2>&1 | tee -a target/coverage/test.log; \ + rc=$${PIPESTATUS[0]}; \ + echo " lcov: $$(ls -la $(CURDIR)/target/coverage/lcov.info 2>&1)"; \ + echo " lcov files under the workspace: $$(find $(CURDIR) -name lcov.info -newer target/coverage/.exclude-re 2>/dev/null | tr '\n' ' ')"; \ + echo " profraw files: $$(find $${CARGO_TARGET_DIR:-$(CURDIR)/target} -name '*.profraw' 2>/dev/null | wc -l)"; \ + if [ "$$rc" -ne 0 ]; then echo "❌ coverage DID NOT MEASURE: the merged report step exited $$rc. No coverage verdict."; exit 1; fi + @# #3839: --ignore-run-fail keeps one failing test from blanking the number (the 2026-09-23 + @# nightly wrote no lcov because of one timing test). Failures are LISTED, not hidden, and + @# every test run here is also run by CI's workspace-test, which fails on them. + @grep -E '^test .* \.\.\. FAILED$$' target/coverage/test.log | sed -e 's/^test //' -e 's/ \.\.\. FAILED$$//' | sort -u > target/coverage/failed-tests.txt || true + @# A test BINARY killed by a signal (earlyoom SIGTERMed aprender-serve at 25.7 GB on yoga, run + @# 35868368976) is swallowed by --ignore-run-fail, and its crate's profile is missing from the + @# lcov: that run printed "76% ... REGRESSION" with the largest crate absent. No verdict then. + @scripts/check_coverage_log_complete.sh target/coverage/test.log @echo "📊 Parsing LCOV for the threshold check..." @# Parse LCOV for line coverage (LH=lines hit, LF=lines found) - @LH=$$(awk -F: '/^LH:/{s+=$$2} END{print s+0}' target/coverage/lcov.info); \ + @if [ ! -s target/coverage/lcov.info ]; then echo "❌ coverage DID NOT MEASURE: no lcov.info was written. No coverage verdict."; exit 1; fi; \ + LH=$$(awk -F: '/^LH:/{s+=$$2} END{print s+0}' target/coverage/lcov.info); \ LF=$$(awk -F: '/^LF:/{s+=$$2} END{print s+0}' target/coverage/lcov.info); \ - if [ "$$LF" -gt 0 ]; then COV_PCT=$$((LH * 100 / LF)); else COV_PCT=0; fi; \ + if [ "$$LF" -eq 0 ]; then echo "❌ coverage DID NOT MEASURE: lcov.info has 0 instrumented lines. No coverage verdict."; exit 1; fi; \ + COV_PCT=$$((LH * 100 / LF)); \ + NFAIL=$$(wc -l < target/coverage/failed-tests.txt); \ echo "TOTAL: $$LH/$$LF lines covered ($${COV_PCT}%)"; \ - echo "TOTAL $$LH $$LF $${COV_PCT}%" > target/coverage/summary.txt; \ + echo "TOTAL $$LH $$LF $${COV_PCT}% failed_tests=$$NFAIL" > target/coverage/summary.txt; \ + if [ "$$NFAIL" -gt 0 ]; then \ + echo "⚠ $$NFAIL test(s) FAILED in the instrumented run (measured anyway; CI workspace-test gates them):"; \ + sed 's/^/ /' target/coverage/failed-tests.txt; \ + sed 's/^/FAILED /' target/coverage/failed-tests.txt >> target/coverage/summary.txt; \ + fi; \ mkdir -p .pmat-metrics || exit 1; \ printf '{"coverage_pct":%s}' "$$COV_PCT" > .pmat-metrics/coverage.result; \ echo " wrote .pmat-metrics/coverage.result ($${COV_PCT}%) for pmat score"; \ - test -f ~/.cargo/config.toml.bak && mv ~/.cargo/config.toml.bak ~/.cargo/config.toml || true; \ if [ "$$COV_PCT" -lt "$(COV_FLOOR)" ]; then \ echo "❌ REGRESSION: coverage $${COV_PCT}% fell below the enforced floor $(COV_FLOOR)%"; \ echo " The floor is the last measured value, so this means coverage went DOWN."; \ @@ -657,12 +782,11 @@ coverage-fast: coverage # gate anything - unlike `coverage`, whose 0% fed the >=95% threshold check. coverage-html: ## Generate HTML + LCOV reports from last coverage run @echo "📊 Generating HTML + LCOV reports..." - @test -f ~/.cargo/config.toml && mv ~/.cargo/config.toml ~/.cargo/config.toml.bak || true + $(COV_REFUSE_GLOBAL_MOLD) @mkdir -p target/coverage @printf '%s' '$(COVERAGE_EXCLUDE_REGEX)' > target/coverage/.exclude-re - @$(COV_CARGO_ENV) cargo llvm-cov report --html --output-dir target/coverage/html --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" - @$(COV_CARGO_ENV) cargo llvm-cov report --lcov --output-path target/coverage/lcov.info --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" - @test -f ~/.cargo/config.toml.bak && mv ~/.cargo/config.toml.bak ~/.cargo/config.toml || true + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py --exclude aprender-gpu) --html --output-dir target/coverage/html --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py --exclude aprender-gpu) --lcov --output-path target/coverage/lcov.info --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" @echo "📍 HTML: target/coverage/html/index.html" # Full coverage: All features (for CI, slower) @@ -670,18 +794,17 @@ coverage-html: ## Generate HTML + LCOV reports from last coverage run coverage-full: ## Full coverage report (all features, CI only) @echo "📊 Running full coverage analysis (all features)..." @which cargo-llvm-cov > /dev/null 2>&1 || { cargo install cargo-llvm-cov --locked || exit 1; } - @test -f ~/.cargo/config.toml && mv ~/.cargo/config.toml ~/.cargo/config.toml.bak || true + $(COV_REFUSE_GLOBAL_MOLD) @mkdir -p target/coverage @printf '%s' '$(COVERAGE_EXCLUDE_REGEX)' > target/coverage/.exclude-re @PROPTEST_CASES=10 QUICKCHECK_TESTS=10 CARGO_BUILD_JOBS=4 \ $(COV_CARGO_ENV) cargo llvm-cov test --no-report --workspace --lib --all-features \ --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" \ -- --skip prop_gbm_expected_value --skip slow --skip heavy --skip benchmark --skip h12_ --skip j2_ - @$(COV_CARGO_ENV) cargo llvm-cov report --html --output-dir target/coverage/html --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" - @$(COV_CARGO_ENV) cargo llvm-cov report --lcov --output-path target/coverage/lcov.info --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py) --html --output-dir target/coverage/html --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py) --lcov --output-path target/coverage/lcov.info --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" @echo "" - @$(COV_CARGO_ENV) cargo llvm-cov report --summary-only --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" - @test -f ~/.cargo/config.toml.bak && mv ~/.cargo/config.toml.bak ~/.cargo/config.toml || true + @$(COV_CARGO_ENV) cargo llvm-cov report $$(python3 scripts/coverage_report_scope.py) --summary-only --ignore-filename-regex "$$(cat target/coverage/.exclude-re)" # Open coverage report in browser coverage-open: ## Open HTML coverage report in browser @@ -1126,7 +1249,12 @@ test-audio-full: ## Run all audio tests including ALSA (if available) # contracts/aprender/binding.yaml. Generated tests: tests/contracts/. # Pre-consolidation `../provable-contracts/` references retired. -PV_BIN := cargo run --release -p aprender-contracts-cli --bin pv -- +# NOT named PV_BIN (PVL-001 EV-4): a makefile assignment overrides an inherited +# environment variable AND is what make exports to recipes, so `PV_BIN := cargo +# run ...` handed scripts/pv_bin.sh the string "cargo run ..." whenever a caller +# exported PV_BIN=/path/to/pv -- the one override pv_bin.sh honours -- and every +# `. scripts/pv_bin.sh` step refused with `not executable: cargo run ...`. +PV_CARGO_RUN := cargo run --release -p aprender-contracts-cli --bin pv -- BINDING := contracts/aprender/binding.yaml CONTRACTS := contracts/softmax-kernel-v1.yaml \ contracts/rmsnorm-kernel-v1.yaml \ @@ -1174,30 +1302,35 @@ contract-validate: ## Validate all kernel contracts (schema + staleness) @echo "Validating kernel contracts..." @for contract in $(CONTRACTS); do \ echo " $$contract"; \ - $(PV_BIN) validate "$$contract" || exit 1; \ + $(PV_CARGO_RUN) validate "$$contract" || exit 1; \ done @echo "Contract validation passed" contract-test: ## Run contract-driven property tests + @set -e @echo "Running contract property tests..." @PROPTEST_CASES=100 cargo test -p aprender-core --test contract_tests @echo "Contract tests passed" contract-audit: ## Audit binding coverage (equations -> implementations) @echo "Running binding audit..." - @for contract in $(CONTRACTS); do \ + @rc=0; for contract in $(CONTRACTS); do \ echo ""; \ - $(PV_BIN) audit "$$contract" --binding $(BINDING); \ + $(PV_CARGO_RUN) audit "$$contract" --binding $(BINDING) || rc=$$?; \ done @echo "" + @if [ "$$rc" -ne 0 ]; then echo "Binding audit FAILED: at least one audit exited non-zero (last rc=$$rc)"; exit "$$rc"; fi @echo "Binding audit complete" +# contract-regen keeps `|| true` ON PURPOSE (PVL-001 EV-4): it is not a gate -- +# it writes .rs.new files for a human to review, and one contract probar cannot +# render must not stop the others being written. contract-regen: ## Regenerate wired test files from contracts @echo "Regenerating contract test files..." @for contract in $(CONTRACTS); do \ name=$$(basename "$$contract" .yaml | sed 's/-kernel-v[0-9]*//;s/-v[0-9]*//'); \ echo " $$name <- $$contract"; \ - $(PV_BIN) probar "$$contract" --binding $(BINDING) > tests/contracts/$${name}_contract.rs.new 2>/dev/null || true; \ + $(PV_CARGO_RUN) probar "$$contract" --binding $(BINDING) > tests/contracts/$${name}_contract.rs.new 2>/dev/null || true; \ done @echo "Regeneration complete (review .rs.new files)" @@ -1352,9 +1485,11 @@ check-siblings: ## Verify sibling repos exist and versions are compatible echo " Remove [patch.crates-io] from .cargo/config.toml"; \ fi -# APR-RELEASE-001 §11.2 (ONT R-6): the five ontology counters move ONLY through -# this target. `--check` is what guard_tree.sh runs on every PR; `--write` is the -# deliberate restamp, and it is the only way a counter is allowed to change. +# APR-RELEASE-001 §11.2 (ONT R-6). Since #3569 the ontology counters are MEASURED, +# never committed: `--check` (guard_tree.sh, every PR) measures them at the +# comparand tree and at the working tree and refuses any move the wrong way. +# `--write` stores DECISIONS only (armed_gates, armed_shapes + the Rust gates' +# foreign keys); it can no longer restamp a counter to hide a regression. .PHONY: ont-ratchet ont-ratchet-check ont-ratchet: @bash scripts/check_ont_ratchet.sh --write @@ -1362,6 +1497,16 @@ ont-ratchet: ont-ratchet-check: @bash scripts/check_ont_ratchet.sh --check +# PVL-001 EV-11 (PMAT-4166): the two `pv lint` ratchets (theorem-pairing, depends-on-present) move ONLY +# through this target, and only DOWN. The gates read contracts/lint-baseline.json and never write it. +# NEVER in CI: a CI job that could rewrite the baseline is a ratchet that turns both ways. +.PHONY: lint-ratchet lint-ratchet-self-test +lint-ratchet: + @bash scripts/lint_ratchet.sh + +lint-ratchet-self-test: + @bash scripts/lint_ratchet.sh --self-test + # ONT-001 §5 ONT-4b2 / R-13 — the out-of-gate SHACL differential oracle. # # NOT a PR check, by the rule that puts it here: `shacl` is 316 crates and pinned at ONE version (ONT-0's diff --git a/README.md b/README.md index 9a73639841..d3c270d6ee 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,8 @@ publishing — all backed by YAML provable contracts that fail CI on drift. | Metric | Count | Source of truth | |-------:|------:|---| -| Workspace crates | **79** workspace crates | `cargo metadata --no-deps` (NOT `ls crates/` — 4 are `exclude`d, 1 has no Cargo.toml) | -| Provable contracts | **1829** provable contracts | `contracts/census.json` `.n_files` — the set `pv lint` walks (`pv census`, ONT-001 ONT-1; regenerated by `make contracts`, written by `make readme-sync`, guarded by `scripts/check_readme_claims.sh`) | +| Workspace crates | **80** workspace crates | `cargo metadata --no-deps` (NOT `ls crates/` — 4 are `exclude`d, 1 has no Cargo.toml) | +| Provable contracts | **1830** provable contracts | `contracts/census.json` `.n_files` — the set `pv lint` walks (`pv census`, ONT-001 ONT-1; regenerated by `make contracts`, written by `make readme-sync`, guarded by `scripts/check_readme_claims.sh`) | | CLI commands | **110** CLI commands | `apr --help` | | Book CLI chapters | **112** chapters | `ls book/src/cli/*.md` | | Book lib chapters | **71** chapters | `ls book/src/lib/*.md` (parity with `pub mod`) | @@ -262,7 +262,7 @@ falsification_tests: prediction: apr validate bad-model.apr exits non-zero ``` -The tree carries 1829 contracts across inference, training, quantization, attention, FFN, +The tree carries 1830 contracts across inference, training, quantization, attention, FFN, tokenization, model formats, CLI safety — and this README itself. ## Migration from old crates diff --git a/book/src/tools/apr-cli.md b/book/src/tools/apr-cli.md index fb63e6097d..cbff5e45a6 100644 --- a/book/src/tools/apr-cli.md +++ b/book/src/tools/apr-cli.md @@ -993,7 +993,7 @@ apr cbtop --refresh 500 # 500ms │ Model: qwen2.5-coder-1.5b-q4_k_m.gguf │ │ Backend: GPU (CUDA) │ ├──────────────────────────────────────────────────────────┤ -│ Throughput: 125.3 tok/s │ +│ Throughput: N.N tok/s │ │ Latency: 8.0 ms/tok │ │ Memory: 1.2 GB / 8.0 GB │ │ Utilization: ████████████░░░░░░░░ 60% │ diff --git a/ci/explicit-test-commands.d/447-apr-cli-surface-binary-pin.cmd b/ci/explicit-test-commands.d/447-apr-cli-surface-binary-pin.cmd new file mode 100644 index 0000000000..08a5c395a5 --- /dev/null +++ b/ci/explicit-test-commands.d/447-apr-cli-surface-binary-pin.cmd @@ -0,0 +1 @@ +cargo test -p apr-cli --test surface_binary_pin diff --git a/ci/explicit-test-commands.d/455-aprender-contracts-cli-ont7-valid-under-gate.cmd b/ci/explicit-test-commands.d/455-aprender-contracts-cli-ont7-valid-under-gate.cmd new file mode 100644 index 0000000000..30cdbfb0ae --- /dev/null +++ b/ci/explicit-test-commands.d/455-aprender-contracts-cli-ont7-valid-under-gate.cmd @@ -0,0 +1 @@ +cargo test -p aprender-contracts-cli --test ont7_valid_under_gate diff --git a/ci/explicit-test-commands.d/456-aprender-contracts-cli-ev11-lint-ratchets.cmd b/ci/explicit-test-commands.d/456-aprender-contracts-cli-ev11-lint-ratchets.cmd new file mode 100644 index 0000000000..7f427a94da --- /dev/null +++ b/ci/explicit-test-commands.d/456-aprender-contracts-cli-ev11-lint-ratchets.cmd @@ -0,0 +1 @@ +cargo test -p aprender-contracts-cli --test ev11_lint_ratchets diff --git a/ci/explicit-test-commands.d/457-aprender-contracts-cli-shapes-vacuity.cmd b/ci/explicit-test-commands.d/457-aprender-contracts-cli-shapes-vacuity.cmd new file mode 100644 index 0000000000..75beb0e63c --- /dev/null +++ b/ci/explicit-test-commands.d/457-aprender-contracts-cli-shapes-vacuity.cmd @@ -0,0 +1 @@ +cargo test -p aprender-contracts-cli --test shapes_vacuity diff --git a/ci/explicit-test-commands.d/480-aprender-contracts-cli-ont4c5-capability-cells.cmd b/ci/explicit-test-commands.d/480-aprender-contracts-cli-ont4c5-capability-cells.cmd new file mode 100644 index 0000000000..633f6310ab --- /dev/null +++ b/ci/explicit-test-commands.d/480-aprender-contracts-cli-ont4c5-capability-cells.cmd @@ -0,0 +1 @@ +cargo test -p aprender-contracts-cli --test ont4c5_capability_cells diff --git a/contracts/alibi-slopes-v1.yaml b/contracts/alibi-slopes-v1.yaml index 688538bf36..816a2e47e6 100644 --- a/contracts/alibi-slopes-v1.yaml +++ b/contracts/alibi-slopes-v1.yaml @@ -55,7 +55,7 @@ proof_obligations: mathlib_imports: - Mathlib.Analysis.SpecialFunctions.Pow.Real notes: 'Exponent -8(h+1)/n < 0 for h >= 0, n > 0; base 2 > 1 so 2^(neg) < 1. - Companion alibi_slope_pos proves 0 < m[h] (strictly positive geometric sequence).' + Companion alibi_slope_pos_real proves 0 < m[h] (strictly positive geometric sequence).' - id: AS-EQ-003 type: equivalence property: Matches ggml reference exponent @@ -89,7 +89,7 @@ verification_summary: theorem: Alibi.alibi_slope_lt_one file: ProvableContracts/Theorems/Alibi/Slopes.lean note: 'exponent -8(h+1)/n < 0 for h>=0, n>0, and base 2>1, so 2^(neg) < 1 - (Real.rpow_lt_one_of_one_lt_of_neg). alibi_slope_pos additionally proves 0 < m[h].' + (Real.rpow_lt_one_of_one_lt_of_neg). alibi_slope_pos_real additionally proves 0 < m[h].' - obligation: Matches ggml reference exponent theorem: Alibi.alibi_slope_ggml file: ProvableContracts/Theorems/Alibi/Slopes.lean @@ -97,7 +97,7 @@ verification_summary: matching llama.cpp soft_max_ext.' notes: 'All 3 obligations proved sorry-free in Lean 4 over the reals (Mathlib Real.rpow). Zero N/A — the contract is fully analytic. Slopes are a positive, - strictly-decreasing geometric sequence: alibi_slope_pos (m[h]>0) and + strictly-decreasing geometric sequence: alibi_slope_pos_real (m[h]>0) and alibi_slope_strict_anti (m[0]>m[1]>...>m[n-1]) additionally proved. Verified via lake env lean ProvableContracts/Theorems/Alibi/Slopes.lean (exit 0, 0 sorry).' kernel_structure: diff --git a/contracts/apr-cli-commands-v1.yaml b/contracts/apr-cli-commands-v1.yaml index dfba96b2dc..bdd7b6c8b1 100644 --- a/contracts/apr-cli-commands-v1.yaml +++ b/contracts/apr-cli-commands-v1.yaml @@ -382,6 +382,13 @@ commands: requires_model: false side_effects: [] + - name: ptx-debug + category: hardware + description: "PTX falsification analysis and FKR test generation (was the aprender-ptx-debug binary)" + requires_model: false + side_effects: [filesystem] + subcommands: [analyze, gen-fkr, version] + - name: ptx-map category: hardware description: "Model-to-PTX source mapping" diff --git a/contracts/apr-cli-publish-v1.yaml b/contracts/apr-cli-publish-v1.yaml index 286aeb97d4..b0275886db 100644 --- a/contracts/apr-cli-publish-v1.yaml +++ b/contracts/apr-cli-publish-v1.yaml @@ -74,7 +74,7 @@ falsification_tests: rule: "publishing goes only through scripts/check_publish_preflight.sh, and the cascade carries no --allow-dirty (F-9)" prediction: "check_publish_preflight.sh --selftest drives R1, R3, R4 and R5 to both verdicts (16 rows); cascade-publish.sh calls the gate before its first upload and its cargo publish line has no --allow-dirty" test: "bash scripts/check_publish_preflight.sh --selftest && grep -q 'scripts/check_publish_preflight.sh' scripts/cascade-publish.sh && ! grep -q -- '--allow-dirty' scripts/cascade-publish.sh" - if_fails: "a dirty, untagged, off-main or NO-GO tree can be uploaded to an immutable registry" + if_fails: "a dirty, untagged, off-release-branch or NO-GO tree can be uploaded to an immutable registry" proof_obligations: - id: ACP-INV-001 diff --git a/contracts/apr-dogfood-coverage-v1.yaml b/contracts/apr-dogfood-coverage-v1.yaml index f4d1558b70..da5092049c 100644 --- a/contracts/apr-dogfood-coverage-v1.yaml +++ b/contracts/apr-dogfood-coverage-v1.yaml @@ -3,7 +3,7 @@ metadata: created: '2026-08-22' author: PAIML Engineering references: - - 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either' + - 'docs/audits/surface_audit.csv — 142 of 830 features gated, across 28 binaries and 14 clusters, measured 2026-08-22 @ 4bbfeb07f; 142 of 832 features gated after the PERF-046 partial re-audit (PR #2742); 143 of 833 features gated after the PP-LLAMA-001 re-audit (GET /v1/effective-config added and swept by the dogfood route enumeration, router.rs rows re-pointed); 144 of 837 features gated after PP-066 L0-1b added the three `apr parity` flag rows (--per-op, --out, --threshold), which the dogfood sweep does not reach -- the denominator grew and the numerator did not; 144 of 838 features gated after #3726 added `apr tokenize encode` (the tokenizer-parity gate''s id source), which the dogfood sweep does not reach either; 144 of 864 features gated after #4219 added one ` --version` row for each of the 26 build-SHA bins (each run and observed printing its SHA), which the dogfood sweep does not reach' - 'docs/audits/surface_audit_elbow.png — k-selection evidence: inertia knee and silhouette over k in [2,45]' - 'scripts/dogfood_cluster.py — the k-sweep that produced the elbow plot (k-selection ONLY; it does not write the columns)' - 'scripts/check_no_cluster_id_keys.sh — T1 enforcement: nothing may key on `cluster_id`' @@ -66,9 +66,9 @@ metadata: overall: command: "python3 scripts/dogfood_baseline.py # section: totals" - rows: 838 + rows: 864 covered: 144 - ratio: 0.1718 # 144/838 -- 17.2%, NOT "about 17%" + ratio: 0.1667 # 144/864 -- 16.7%, NOT "about 17%" binaries: 28 binaries_at_zero_coverage: 27 clusters: 14 @@ -283,7 +283,7 @@ equations: codomain: bool invariants: - "T2. Cluster coverage is a PROXY: one gate in a 95-member cluster is 1%, not covered" - - "5/14 clusters (35.7%) and 144/838 features (17.2%) are reported together, always" + - "5/14 clusters (35.7%) and 144/864 features (16.7%) are reported together, always" - "Enforced mechanically by enforce_pairing(); deleting the feature fraction from the emitter turns the gate RED" - "This is the single most important trap: reporting the proxy alone is how a coverage gate becomes theatre while looking stricter" @@ -672,7 +672,7 @@ falsification_tests: Every emitted line stating a CLUSTER-level ratio also states the underlying FEATURE-level ratio, on EVERY channel one can be emitted through: this gate's report, the receipt, scripts/dogfood_baseline.py and - .claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 838 features (17.2%) is the thing. + .claude/skills/apr-dogfood/SKILL.md. 5 of 14 clusters (35.7%) is a proxy and 144 of 864 features (16.7%) is the thing. One gate in a 95-member cluster is 1%. The rule keys on the NUMBER -- a denominator equal to the cluster count -- not on a phrase, because a phrase can be reworded and a denominator cannot; diff --git a/contracts/apr-version-traceability-v1.yaml b/contracts/apr-version-traceability-v1.yaml index 8c5bf391cd..1cd23cee4a 100644 --- a/contracts/apr-version-traceability-v1.yaml +++ b/contracts/apr-version-traceability-v1.yaml @@ -1,11 +1,12 @@ metadata: - version: 1.1.0 + version: 1.2.0 created: '2026-04-05' author: PAIML Engineering description: "apr --version traceability contract — the version output must always contain an informative build identifier; never the bare '(unknown)' sentinel when a CARGO_PKG_VERSION or any alternative identifier is available; AND the embedded git SHA must match the actual HEAD of the source tree at build time (including in git worktrees)" references: - "paiml/aprender#597 (Version string shows (unknown) instead of git hash)" - "paiml/aprender#1862 (build.rs misses HEAD changes in worktrees because ../../.git is a file pointer, not a directory)" + - "paiml/aprender#4110 (crates.io installs print +no-git although the tarball's .cargo_vcs_info.json carries the commit)" registry: true tags: - cli @@ -36,11 +37,13 @@ equations: - "Common sentinel forms are forbidden: (unknown), 0000000, , null" fallback_hierarchy: - formula: "APR_GIT_SHA = override ∨ git_hash ∨ committed_sha ∨ 'v{version}+no-git'" + formula: "APR_GIT_SHA = override ∨ vcs_info_sha ∨ git_hash ∨ committed_sha ∨ 'v{version}+no-git'" domain: "build-time environment" codomain: "APR_GIT_SHA environment variable" invariants: - "Env var APR_GIT_SHA_OVERRIDE, if set, wins (for CI/release)" + - "Otherwise: read git.sha1 from the package's .cargo_vcs_info.json (written by cargo package into every published crate), short 9-hex form, -dirty suffix when cargo recorded a dirty tree; a value that is not 40 hex is ignored (#4110)" + - "The vcs rung precedes git rev-parse: an unpacked crate may sit inside an UNRELATED repository whose HEAD is not the crate's commit" - "Otherwise: try git rev-parse --short HEAD" - "Otherwise: try reading crates/apr-cli/.git-sha committed file" - "Otherwise: emit 'v{CARGO_PKG_VERSION}+no-git' as informative fallback" @@ -89,10 +92,15 @@ falsification_tests: prediction: "When built without git context, apr --version output contains the package version string" test: "apr --version 2>&1 | grep -qE '0\\.[1-9][0-9]*\\.[0-9]+'" if_fails: "Version string missing entirely" + - id: FALSIFY-VERSION-005 + rule: "a packaged apr-cli (no .git of its own) names the commit in its .cargo_vcs_info.json, even inside a foreign repository" + prediction: "The shipped build.rs, compiled and run as cargo runs it on a package dir holding the byte-exact apr-cli-0.69.1 vcs file, emits APR_GIT_SHA=d8a6df53a; inside an unrelated repo it still does; with the vcs rung removed or placed behind git rev-parse the table goes RED" + test: "cargo test -p aprender-build-sha --lib emit_tests" + if_fails: "crates.io installs regress to '+no-git' (0.68.1/0.69.1 shape), or report an enclosing repository's HEAD" - id: FALSIFY-VERSION-004 rule: "build.rs registers worktree-aware rerun-if-changed triggers" prediction: "build.rs invokes `git rev-parse --git-dir` and `--git-common-dir` (not a hardcoded `../../.git/HEAD` path)" - test: "grep -qE 'rev-parse.*--git-(dir|common-dir)' crates/apr-cli/build.rs && ! grep -qE '\\.\\./\\.\\./\\.git/HEAD' crates/apr-cli/build.rs" + test: "grep -qE 'rev-parse.*--git-(dir|common-dir)' crates/aprender-build-sha/src/lib.rs && ! grep -qE '\\.\\./\\.\\./\\.git/HEAD' crates/aprender-build-sha/src/lib.rs crates/apr-cli/build.rs" if_fails: "build.rs still relies on a hardcoded relative .git/HEAD path; worktree HEAD changes won't trigger rebuild" kani_harnesses: diff --git a/contracts/aprender/binding.yaml b/contracts/aprender/binding.yaml index 18d9f39a96..8121be678a 100644 --- a/contracts/aprender/binding.yaml +++ b/contracts/aprender/binding.yaml @@ -1,6 +1,162 @@ version: 1.0.0 target_crate: aprender bindings: +- contract: adamw-kernel-v1.yaml + equation: weight_update + module_path: aprender::nn::optim::rm_sprop + function: step + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/optim/rm_sprop.rs:97 had no binding row; row added when the binding moved in-tree' +- contract: batchnorm-kernel-v1.yaml + equation: batchnorm_train + module_path: aprender::nn::normalization + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/normalization/mod.rs:391 had no binding row; row added when the binding moved in-tree' +- contract: cma-es-kernel-v1.yaml + equation: mean_update + module_path: aprender::metaheuristics::cmaes_include_01 + function: optimize + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/metaheuristics/cmaes_include_01.rs:5 had no binding row; row added when the binding moved in-tree' +- contract: cma-es-kernel-v1.yaml + equation: sample + module_path: aprender::metaheuristics::cmaes + function: sample + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/metaheuristics/cmaes.rs:274 had no binding row; row added when the binding moved in-tree' +- contract: conv1d-kernel-v1.yaml + equation: conv1d + module_path: aprender::nn::conv + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/conv/mod.rs:292 had no binding row; row added when the binding moved in-tree' +- contract: cross-entropy-kernel-v1.yaml + equation: cross_entropy + module_path: aprender::nn::loss + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/loss.rs:257 had no binding row; row added when the binding moved in-tree' +- contract: cross-entropy-kernel-v1.yaml + equation: log_softmax + module_path: aprender::nn::functional + function: log_softmax + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/functional.rs:307 had no binding row; row added when the binding moved in-tree' +- contract: embedding-algebra-v1.yaml + equation: embedding_lookup + module_path: aprender::models::qwen2 + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/models/qwen2/mod.rs:121 had no binding row; row added when the binding moved in-tree' +- contract: f16-conversion-v1.yaml + equation: f16_to_f32_bias + module_path: aprender::format::gguf::dequant + function: f16_to_f32 + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/format/gguf/dequant.rs:18 had no binding row; row added when the binding moved in-tree' +- contract: gqa-kernel-v1.yaml + equation: gqa + module_path: aprender::nn::transformer + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/transformer/mod.rs:409 had no binding row; row added when the binding moved in-tree' +- contract: kmeans-kernel-v1.yaml + equation: assignment + module_path: aprender::cluster::kmeans_impl + function: predict + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/cluster/kmeans_impl.rs:406 had no binding row; row added when the binding moved in-tree' +- contract: kmeans-kernel-v1.yaml + equation: update + module_path: aprender::cluster::kmeans_impl + function: fit + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/cluster/kmeans_impl.rs:362 had no binding row; row added when the binding moved in-tree' +- contract: layernorm-kernel-v1.yaml + equation: layernorm + module_path: aprender::nn::normalization + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/normalization/mod.rs:131 had no binding row; row added when the binding moved in-tree' +- contract: matmul-kernel-v1.yaml + equation: matmul + module_path: aprender::autograd::ops::activation + function: matmul + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/activation.rs:272 had no binding row; row added when the binding moved in-tree' +- contract: rmsnorm-kernel-v1.yaml + equation: rmsnorm + module_path: aprender::nn::normalization::group_norm + function: forward + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/normalization/group_norm.rs:348 had no binding row; row added when the binding moved in-tree' +- contract: rope-kernel-v1.yaml + equation: rope + module_path: aprender::nn::transformer::attention_helpers + function: apply + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/transformer/attention_helpers.rs:268 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: additive_attention_mask + module_path: aprender::autograd::ops::masking + function: additive_attention_mask + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/masking.rs:43 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: apply_additive_mask + module_path: aprender::nn::transformer::positional_encoding + function: add_mask + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/transformer/positional_encoding.rs:476 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: cosine_similarity_rows + module_path: aprender::autograd::ops::similarity + function: cosine_similarity_rows + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/similarity.rs:55 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: embedding_gather + module_path: aprender::autograd::ops::embedding + function: embedding_gather + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/embedding.rs:38 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: gelu_exact + module_path: aprender::autograd::ops::activation + function: gelu_exact + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/activation.rs:181 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: l2_normalize_rows + module_path: aprender::autograd::ops::normalize + function: l2_normalize_rows + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/normalize.rs:56 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: masked_mean_pool + module_path: aprender::autograd::ops::pooling + function: masked_mean_pool + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/pooling.rs:50 had no binding row; row added when the binding moved in-tree' +- contract: setfit-encoder-conformance-v1.yaml + equation: mse_loss + module_path: aprender::autograd::ops::similarity + function: mse_loss + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/autograd/ops/similarity.rs:196 had no binding row; row added when the binding moved in-tree' +- contract: silu-kernel-v1.yaml + equation: sigmoid + module_path: aprender::nn::functional + function: sigmoid + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/functional.rs:58 had no binding row; row added when the binding moved in-tree' +- contract: silu-kernel-v1.yaml + equation: silu + module_path: aprender::nn::functional + function: silu + status: implemented + notes: '#4369: #[contract] site at crates/aprender-core/src/nn/functional.rs:90 had no binding row; row added when the binding moved in-tree' - contract: softmax-kernel-v1.yaml equation: softmax module_path: aprender::nn::functional::softmax diff --git a/contracts/batuta/binding.yaml b/contracts/batuta/binding.yaml new file mode 100644 index 0000000000..6d3c423643 --- /dev/null +++ b/contracts/batuta/binding.yaml @@ -0,0 +1,316 @@ +version: 1.0.0 +target_crate: batuta +critical_path: +- transpile +- analyze +- build_transpiler +- recommend_transpiler +- cmd_transpile +bindings: +- contract: builder-pattern-v1.yaml + equation: transpile + function: build_transpiler_args + module_path: batuta::src::cli::mod::build_transpiler_args + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: cmd_transpile + module_path: batuta::src::cli::pipeline_cmds_transpile::cmd_transpile + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: recommend_transpiler + module_path: batuta::src::types::recommend_transpiler + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: has_transpiler + module_path: batuta::src::tools::has_transpiler + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: get_transpiler_for_language + module_path: batuta::src::tools::get_transpiler_for_language + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: transpile_python + module_path: batuta::src::tools::transpile_python + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: transpile_shell + module_path: batuta::src::tools::transpile_shell + status: implemented +- contract: configuration-v1.yaml + equation: transpile + function: transpile_c_cpp + module_path: batuta::src::tools::transpile_c_cpp + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_numpy_usage + module_path: batuta::src::pipeline_analysis::analyze_numpy_usage + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_sklearn_usage + module_path: batuta::src::pipeline_analysis::analyze_sklearn_usage + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_pytorch_usage + module_path: batuta::src::pipeline_analysis::analyze_pytorch_usage + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_contract_gaps + module_path: batuta::src::bug_hunter::contracts::analyze_contract_gaps + status: implemented +- contract: linear-models-v1.yaml + equation: analyze + function: analyze_model_parity_gaps + module_path: batuta::src::bug_hunter::model_parity::analyze_model_parity_gaps + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze + module_path: batuta::src::recipes::benchmark::analyze + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_optimizations + module_path: batuta::src::pipeline::stages::optimization::analyze_optimizations + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: cmd_analyze + module_path: batuta::src::cli::pipeline_cmds::cmd_analyze + status: implemented +- contract: gpu-decode-profiling-v1.yaml + equation: analyze + function: analyze_code + module_path: batuta::src::wasm::analyze_code + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_quality + module_path: batuta::src::tools::analyze_quality + status: implemented +- contract: configuration-v1.yaml + equation: analyze + function: analyze_dependencies + module_path: batuta::src::parf::analyze_dependencies + status: implemented +- contract: fused-qkv-projection-v1.yaml + equation: analyze + function: analyze_project + module_path: batuta::src::analyzer::analyze_project + status: implemented +# --- CCX-RS-inspired features (2026-04-02) --- +- contract: provider-routing-v1.yaml + equation: privacy_enforcement + function: route_to_provider + module_path: batuta::src::agent::driver::routing::route_to_provider + status: pending +- contract: provider-routing-v1.yaml + equation: failover_cascade + function: failover_next + module_path: batuta::src::agent::driver::routing::failover_next + status: pending +- contract: provider-routing-v1.yaml + equation: cost_budget + function: estimate_cost + module_path: batuta::src::agent::driver::remote::estimate_cost + status: pending +- contract: provider-routing-v1.yaml + equation: backoff_jitter + function: delay_for + module_path: batuta::src::agent::driver::remote::backoff::delay_for + status: pending +- contract: provider-routing-v1.yaml + equation: format_translation + function: to_openai + module_path: batuta::src::agent::driver::remote::translator::to_openai + status: pending +- contract: agent-loop-v1.yaml + equation: loop_termination + function: run_loop + module_path: batuta::src::agent::run_loop + status: pending +- contract: agent-loop-v1.yaml + equation: state_machine + function: transition + module_path: batuta::src::agent::state::transition + status: pending +- contract: agent-loop-v1.yaml + equation: context_compaction + function: compact + module_path: batuta::src::agent::context::compact + status: pending +- contract: agent-loop-v1.yaml + equation: sandbox_enforcement + function: check_sandbox + module_path: batuta::src::agent::sandbox::check_sandbox + status: pending +- contract: agent-loop-v1.yaml + equation: parallel_tool_safety + function: execute_all + module_path: batuta::src::agent::tool::execute_all + status: pending +- contract: agent-loop-v1.yaml + equation: session_crash_recovery + function: resume + module_path: batuta::src::agent::session::resume + status: pending +- contract: agent-loop-v1.yaml + equation: hook_ordering + function: execute_with_hooks + module_path: batuta::src::agent::hook::execute_with_hooks + status: pending +# --- Presentar/Probar UX contracts (2026-04-02) --- +- contract: agent-ux-v1.yaml + equation: streaming_responsiveness + function: push_token + module_path: batuta::src::agent::tui::streaming::push_token + status: pending +- contract: agent-ux-v1.yaml + equation: frame_budget + function: render_frame + module_path: batuta::src::agent::tui::layout::render_frame + status: pending +- contract: agent-ux-v1.yaml + equation: brick_verification + function: can_render + module_path: batuta::src::agent::brick::can_render + status: pending +- contract: agent-ux-v1.yaml + equation: cost_display_accuracy + function: update_cost + module_path: batuta::src::agent::tui::cost::update_cost + status: pending +- contract: agent-ux-v1.yaml + equation: layout_correctness + function: compute_layout + module_path: batuta::src::agent::tui::layout::compute_layout + status: pending +- contract: agent-ux-v1.yaml + equation: state_machine_validity + function: transition + module_path: batuta::src::agent::state::transition + status: pending +# --- apr code contracts (2026-04-02) --- +- contract: apr-code-v1.yaml + equation: sovereignty_guarantee + function: enforce_offline + module_path: batuta::src::agent::sandbox::enforce_offline + status: pending +- contract: apr-code-v1.yaml + equation: tool_safety + function: execute_tool_checked + module_path: batuta::src::agent::tool::execute_tool_checked + status: pending +- contract: apr-code-v1.yaml + equation: session_integrity + function: resume_session + module_path: batuta::src::agent::session::resume_session + status: pending +- contract: apr-code-v1.yaml + equation: apr_md_compliance + function: load_apr_md + module_path: batuta::src::agent::config::load_apr_md + status: pending +- contract: apr-code-v1.yaml + equation: startup_latency + function: run_apr_code + module_path: batuta::src::agent::run_apr_code + status: pending +# --- Oracle CLI contracts (2026-04-03) --- +- contract: cli-oracle-v1.yaml + equation: dispatch_correctness + function: dispatch_oracle + module_path: batuta::src::cli::oracle::mod::dispatch_oracle + status: pending + notes: Every oracle subcommand variant has a dispatch handler +- contract: cli-oracle-v1.yaml + equation: rag_query_correctness + function: rag_search + module_path: batuta::src::cli::oracle::rag::rag_search + status: pending + notes: RAG results sorted by BM25 score descending, empty query returns empty +- contract: cli-oracle-v1.yaml + equation: index_freshness + function: check_freshness + module_path: batuta::src::cli::oracle::rag_index::check_freshness + status: pending + notes: Stale index warns but proceeds, missing index triggers auto-build +# --- HTTP API contracts (2026-04-04, PMAT-189) --- +- contract: http-api-v1.yaml + equation: body_schema_compliance + function: build_openai_body + module_path: batuta::src::agent::driver::apr_serve::build_openai_body + status: implemented + notes: AprServeDriver builds OpenAI-compatible JSON request bodies +- contract: http-api-v1.yaml + equation: max_tokens_cap + function: build_openai_body + module_path: batuta::src::agent::driver::apr_serve::build_openai_body + status: implemented + notes: max_tokens capped at 1024 (PMAT-170) +- contract: http-api-v1.yaml + equation: thinking_block_strip + function: strip_thinking_blocks + module_path: batuta::src::agent::driver::apr_serve::strip_thinking_blocks + status: implemented + notes: Removes ... from Qwen3 responses (PMAT-180) +# --- Session contracts (2026-04-04, PMAT-189) --- +- contract: session-v1.yaml + equation: jsonl_roundtrip + function: append_messages + module_path: batuta::src::agent::session::append_messages + status: implemented + notes: JSONL append-only persistence +- contract: session-v1.yaml + equation: manifest_serde + function: new + module_path: batuta::src::agent::session::Session::new + status: implemented + notes: Session manifest serialization +- contract: session-v1.yaml + equation: age_filter + function: find_recent_for_cwd + module_path: batuta::src::agent::session::find_recent_for_cwd + status: implemented + notes: 24h age filter (PMAT-165) +# --- Tokenizer contracts (2026-04-04, PMAT-189) --- +- contract: tokenizer-v1.yaml + equation: deterministic_encode + function: encode + module_path: batuta::src::agent::driver::validate::tokenizer_encode + status: implemented + notes: Tokenizer used for context window management +- contract: tokenizer-v1.yaml + equation: vocab_size_bound + function: encode + module_path: batuta::src::agent::driver::validate::tokenizer_encode + status: implemented + notes: Token IDs must be within vocabulary bounds +# --- Model discovery contracts (2026-04-04, PMAT-188) --- +- contract: apr-model-discovery-v1.yaml + equation: search_order + function: discover_model + module_path: batuta::src::agent::manifest::discover_model + status: implemented + notes: ~/.apr/models/ > ~/.cache/huggingface/ > ./models/ +- contract: apr-model-discovery-v1.yaml + equation: sort_priority + function: discover_model + module_path: batuta::src::agent::manifest::discover_model + status: implemented + notes: valid > mtime > APR (PMAT-185) +- contract: apr-model-discovery-v1.yaml + equation: jidoka_validation + function: is_valid_model_file + module_path: batuta::src::agent::driver::validate::is_valid_model_file + status: implemented + notes: Invalid APR deprioritized behind valid GGUF (PMAT-150) diff --git a/contracts/binding.yaml b/contracts/binding.yaml index 1391a732f4..2991935296 100644 --- a/contracts/binding.yaml +++ b/contracts/binding.yaml @@ -476,10 +476,11 @@ bindings: # --- apr-model-graph-v1 (full LLM forward pass) --- - contract: apr-model-graph-v1.yaml equation: forward_pass_completeness - module_path: aprender::inference::forward - function: forward_pass + module_path: realizar::gguf + function: OwnedQuantizedModel::forward + signature: 'fn forward(&self, token_ids: &[u32]) -> Result>' status: implemented - notes: Full forward pass from tokens to logits + notes: Full forward pass from tokens to logits (crates/aprender-serve/src/gguf/inference/forward/forward_fused_q4k.rs); returns last-position logits [V], not [B,S,V]. Was ghost aprender::inference::forward::forward_pass (#4094) - contract: apr-model-graph-v1.yaml equation: attention_mechanism module_path: aprender::nn::transformer @@ -666,7 +667,7 @@ bindings: status: implemented notes: >- S8, the four jidoka parameters. guard_mutation_score is REPRODUCIBLE: - scripts/mutate-guard.sh is committed and its DERIVED set is 233/233 over + scripts/mutate-guard.sh is committed and its DERIVED set is 237/237 over scripts/check_pr_review_receipt.sh. THE SET SIZE IS DERIVED AND CERTAIN; THE KILL COUNT ON THIS MERGED TREE IS PENDING, and is the `pr-review-receipt` job's Arm 3. 215/215 was measured on PRREV-015's tree and 185/185 on PRREV-013's, and neither diff --git a/contracts/contracts.nt b/contracts/contracts.nt index a958ebaa1e..c4d057aeea 100644 --- a/contracts/contracts.nt +++ b/contracts/contracts.nt @@ -8377,6 +8377,18 @@ . "contracts/online-softmax-v1.yaml"^^ . "online-softmax-v1"^^ . + . + . + . + . + . + "gguf"^^ . + "contracts/ont-capability-cells-v1.yaml"^^ . + "ont-capability-cells-v1"^^ . + "pattern"^^ . + "ont-capability-cells"^^ . + "active"^^ . + "1.0.0"^^ . . . . @@ -8442,6 +8454,16 @@ "ont-sigma"^^ . "active"^^ . "1.0.0"^^ . + . + . + . + . + "contracts/ont-valid-under-v1.yaml"^^ . + "ont-valid-under-v1"^^ . + "pattern"^^ . + "ont-valid-under"^^ . + "active"^^ . + "1.0.0"^^ . . . "contracts/ont-verdict-lattice-v1.yaml"^^ . @@ -8676,6 +8698,15 @@ "contracts/pv-version-identity-v1.yaml"^^ . "pv-version-identity-v1"^^ . "pattern"^^ . + . + . + . + "contracts/pvl-lint-ratchets-v1.yaml"^^ . + "pvl-lint-ratchets-v1"^^ . + "pattern"^^ . + "pvl-lint-ratchets"^^ . + "active"^^ . + "1.0.0"^^ . . . "contracts/q2k-dequant-parity-v1.yaml"^^ . @@ -12541,15 +12572,15 @@ "ProvableContracts.Theorems.Alibi.Slopes"^^ . "alibi_slope_lt_one"^^ . "true"^^ . - . - . - "grounded"^^ . - "Alibi"^^ . - "crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean"^^ . - . - "ProvableContracts.Theorems.Alibi.Slopes"^^ . - "alibi_slope_pos"^^ . - "true"^^ . + . + . + "grounded"^^ . + "Alibi"^^ . + "crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean"^^ . + . + "ProvableContracts.Theorems.Alibi.Slopes"^^ . + "alibi_slope_pos_real"^^ . + "true"^^ . . . "grounded"^^ . diff --git a/contracts/entrenar/binding.yaml b/contracts/entrenar/binding.yaml new file mode 100644 index 0000000000..7b13fc37d4 --- /dev/null +++ b/contracts/entrenar/binding.yaml @@ -0,0 +1,369 @@ +version: 1.0.0 +target_crate: entrenar +bindings: +- contract: backward-pass-v1.yaml + equation: backward + module_path: entrenar::transformer::cuda_block + function: backward + status: partial + notes: '#4369: #[contract] site at crates/aprender-train/src/transformer/cuda_block.rs:1275 (feature cuda) had no binding row, and contracts/backward-pass-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: apr-checkpoint-v1.yaml + equation: save_checkpoint + module_path: entrenar::finetune::classify_trainer + function: save_checkpoint + signature: 'fn save_checkpoint(&self, path: &Path) -> Result<()>' + status: implemented + notes: APR v1.2.0 format with adapter + training state +- contract: apr-checkpoint-v1.yaml + equation: load_checkpoint + module_path: entrenar::finetune::classify_trainer + function: load_checkpoint + signature: 'fn load_checkpoint(&mut self, path: &Path) -> Result<()>' + status: implemented + notes: Restores optimizer state, epoch, step counters +- contract: apr-checkpoint-v1.yaml + equation: apr_writer + module_path: aprender::serialization::apr + function: AprWriter::write + signature: 'fn write(&self, path: &Path, tensors: &[(&str, &Array1)]) -> Result<()>' + status: implemented + notes: Cross-crate; aprender provides APR format I/O +- contract: apr-checkpoint-v1.yaml + equation: apr_reader + module_path: aprender::serialization::apr + function: AprReader::read + signature: 'fn read(path: &Path) -> Result)>>' + status: implemented + notes: Cross-crate; aprender provides APR format I/O +- contract: attention-head-extraction-v1.yaml + equation: forward_with_lora + module_path: entrenar::transformer::attention + function: MultiHeadAttention::forward_with_lora + signature: 'fn forward_with_lora(&self, x: &Tensor, mask: Option<&Tensor>) -> Tensor' + status: implemented + notes: 'KAIZEN-016: 99.2% reduction in per-head allocations' +- contract: attention-head-extraction-v1.yaml + equation: head_extraction + module_path: entrenar::transformer::attention + function: MultiHeadAttention::forward + signature: 'fn forward(&self, x: &Tensor, mask: Option<&Tensor>) -> Tensor' + status: implemented + notes: Pre-allocated head buffers; zero per-head heap alloc +- contract: cuda-classify-training-v1.yaml + equation: gpu_forward + module_path: entrenar::transformer::cuda_block + function: CudaTransformerBlock::gpu_forward + signature: 'fn gpu_forward(&self, hidden: &CudaBuffer) -> CudaBuffer' + status: implemented + notes: GPU forward pass; parity within 1e-3 of CPU +- contract: cuda-classify-training-v1.yaml + equation: weight_roundtrip + module_path: entrenar::autograd::cuda_training + function: weight_roundtrip + signature: 'fn weight_roundtrip(cpu: &[f32], device: &CudaDevice) -> Vec' + status: pending + notes: function not found in source +- contract: cuda-classify-training-v1.yaml + equation: device_dispatch + module_path: entrenar::autograd::cuda_training + function: ComputeDevice::auto_detect + signature: fn auto_detect() -> ComputeDevice + status: implemented + notes: Auto-detects CUDA, wgpu, or CPU fallback +- contract: distributed-training-v1.yaml + equation: gradient_allreduce + module_path: entrenar::finetune::data_parallel + function: DataParallelCoordinator::allreduce + signature: 'fn allreduce(&self, grads: &mut [Tensor]) -> Result<()>' + status: implemented + notes: Phase 1 complete (multi-GPU); Phase 2 in progress (multi-node) +- contract: distributed-training-v1.yaml + equation: sharding + module_path: entrenar::finetune::data_parallel + function: DataParallelCoordinator::shard_data + signature: 'fn shard_data(&self, dataset: &[Sample]) -> Vec>' + status: implemented + notes: function not found in source +- contract: distributed-training-v1.yaml + equation: weighted_loss + module_path: entrenar::finetune::data_parallel + function: weighted_loss_aggregate + signature: 'fn weighted_loss_aggregate(losses: &[f32], counts: &[usize]) -> f32' + status: pending + notes: function not found in source +- contract: distributed-training-v1.yaml + equation: swiglu_ffn + module_path: entrenar::transformer::cuda_block + function: CudaTransformerBlock::swiglu_ffn + signature: 'fn swiglu_ffn(&self, hidden: &CudaBuffer) -> CudaBuffer' + status: implemented + notes: function not found in source +- contract: distributed-training-v1.yaml + equation: lora_gradient_size + module_path: entrenar::finetune::data_parallel + function: lora_gradient_byte_size + signature: 'fn lora_gradient_byte_size(rank: usize, hidden: usize, layers: usize) + -> usize' + status: pending + notes: function not found in source +- contract: lora-gradient-flow-v1.yaml + equation: transpose_tracked + module_path: entrenar::autograd::ops::matmul + function: transpose_tracked + signature: 'fn transpose_tracked(x: &Tensor) -> Tensor' + status: implemented + notes: 'KAIZEN-018: tracked transpose for LoRA gradient flow' +- contract: lora-gradient-flow-v1.yaml + equation: lora_forward + module_path: entrenar::transformer::attention + function: MultiHeadAttention::forward_with_lora + signature: 'fn forward_with_lora(&self, x: &Tensor, mask: Option<&Tensor>) -> Tensor' + status: implemented + notes: wgpu LoRA gradients restored; 5K to 5.9M trainable params +- contract: qlora-hyperparameters-v1.yaml + equation: learning_rate_scaling + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::learning_rate + signature: fn learning_rate(&self) -> f32 + status: implemented + notes: HP design pending; contract specifies constraints +- contract: qlora-hyperparameters-v1.yaml + equation: effective_batch_size + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::effective_batch_size + signature: fn effective_batch_size(&self) -> usize + status: implemented + notes: function not found in source +- contract: qlora-hyperparameters-v1.yaml + equation: lora_alpha_ratio + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::lora_alpha + signature: fn lora_alpha(&self) -> f32 + status: implemented + notes: function not found in source +- contract: qlora-hyperparameters-v1.yaml + equation: seq_len_from_data + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::max_seq_len + signature: fn max_seq_len(&self) -> usize + status: implemented + notes: Derived from P99 of training data lengths +- contract: qlora-hyperparameters-v1.yaml + equation: warmup_fraction + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::warmup_steps + signature: fn warmup_steps(&self) -> usize + status: implemented + notes: function not found in source +- contract: qlora-hyperparameters-v1.yaml + equation: gradient_clip_bound + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::gradient_clip_norm + signature: fn gradient_clip_norm(&self) -> f32 + status: implemented + notes: function not found in source +- contract: qlora-hyperparameters-v1.yaml + equation: epoch_count_imbalanced + module_path: entrenar::finetune::qlora_config + function: QLoraConfig::epochs + signature: fn epochs(&self) -> usize + status: implemented + notes: 3-10 epochs; more for imbalanced datasets +- contract: tensor-rc-data-v1.yaml + equation: tensor_clone + module_path: entrenar::autograd::tensor + function: Tensor::clone + signature: fn clone(&self) -> Tensor + status: implemented + notes: 'KAIZEN-019: Rc bump (O(1)) instead of 16GB deep copies' +- contract: tensor-rc-data-v1.yaml + equation: data_read + module_path: entrenar::autograd::tensor + function: Tensor::data + signature: fn data(&self) -> &Array1 + status: implemented + notes: Shared read access via Rc +- contract: tensor-rc-data-v1.yaml + equation: data_mut + module_path: entrenar::autograd::tensor + function: Tensor::data_mut + signature: fn data_mut(&mut self) -> &mut Array1 + status: implemented + notes: Copy-on-write via Rc::make_mut +- contract: wgpu-resident-weights-v1.yaml + equation: with_resident_weights + module_path: entrenar::transformer::wgpu_block + function: WgpuForwardPass::with_resident_weights + signature: 'fn with_resident_weights(weights: &GpuWeights) -> WgpuForwardPass' + status: implemented + notes: 'KAIZEN-015: one-time FFN weight upload; frozen during QLoRA' +- contract: wgpu-resident-weights-v1.yaml + equation: import_buffer + module_path: trueno::backends::gpu::batch + function: GpuCommandBatch::import_buffer + signature: 'fn import_buffer(&mut self, buffer: &GpuBuffer)' + status: implemented + notes: function not found in source +- contract: wgpu-resident-weights-v1.yaml + equation: vram_budget + module_path: entrenar::finetune::classify_pipeline + function: estimate_wgpu_vram + signature: 'fn estimate_wgpu_vram(model_params: usize) -> usize' + status: implemented + notes: function not found in source +- contract: cma-es-kernel-v1.yaml + equation: sample + module_path: entrenar + function: sample + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: temperature + module_path: entrenar + function: temperature + status: implemented + notes: Auto-inferred by pv infer +- contract: silu-kernel-v1.yaml + equation: sigmoid + module_path: entrenar + function: sigmoid + status: implemented + notes: Auto-inferred by pv infer +- contract: random-forest-v1.yaml + equation: predict + module_path: entrenar + function: predict + status: implemented + notes: Auto-inferred by pv infer +- contract: metrics-classification-v1.yaml + equation: accuracy + module_path: entrenar + function: accuracy + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: greedy + module_path: entrenar + function: greedy + status: implemented + notes: Auto-inferred by pv infer +- contract: metrics-classification-v1.yaml + equation: confusion_matrix + module_path: entrenar + function: confusion_matrix + status: implemented + notes: Auto-inferred by pv infer +- contract: metrics-ranking-v1.yaml + equation: ndcg_at_k + module_path: entrenar + function: ndcg_at_k + status: implemented + notes: Auto-inferred by pv infer +- contract: layernorm-kernel-v1.yaml + equation: statistics + module_path: entrenar + function: statistics + status: implemented + notes: Auto-inferred by pv infer +- contract: kmeans-kernel-v1.yaml + equation: update + module_path: entrenar + function: update + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: top_k + module_path: entrenar + function: top_k + status: implemented + notes: Auto-inferred by pv infer +- contract: metrics-regression-v1.yaml + equation: rmse + module_path: entrenar + function: rmse + status: implemented + notes: Auto-inferred by pv infer +- contract: int8-symmetric-quant-v1.yaml + equation: quantize + module_path: entrenar + function: quantize + status: implemented + notes: Auto-inferred by pv infer +- contract: model-config-algebra-v1.yaml + equation: bounds + module_path: entrenar + function: bounds + status: implemented + notes: Auto-inferred by pv infer +- contract: gated-delta-net-v1.yaml + equation: output + module_path: entrenar + function: output + status: implemented + notes: Auto-inferred by pv infer +- contract: q4k-q6k-superblock-v1.yaml + equation: total_bytes + module_path: entrenar + function: total_bytes + status: implemented + notes: Auto-inferred by pv infer +- contract: attention-kernel-v1.yaml + equation: attention + module_path: entrenar + function: attention + status: implemented + notes: Auto-inferred by pv infer +- contract: transpose-kernel-v1.yaml + equation: transpose + module_path: entrenar + function: transpose + status: implemented + notes: Auto-inferred by pv infer +- contract: matmul-kernel-v1.yaml + equation: matmul + module_path: entrenar + function: matmul + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: relu + module_path: entrenar + function: relu + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: gelu + module_path: entrenar + function: gelu + status: implemented + notes: Auto-inferred by pv infer +- contract: wgpu-transformer-trainer-v1.yaml + equation: C-WGPU-TRAIN-001 + module_path: entrenar::train::transformer_trainer::wgpu_trainer + function: full_train_step + signature: 'fn full_train_step(&mut self, hidden: &[f32], targets: &[u32], model: + &mut WgpuModelState) -> Result<(f32, f32), String>' + status: implemented + notes: 36-layer Qwen3-4B forward + lm_head + loss + backward + AdamW on WGPU +- contract: wgpu-transformer-trainer-v1.yaml + equation: C-WGPU-CKPT-001 + module_path: entrenar::train::transformer_trainer::wgpu_trainer + function: save_checkpoint + signature: 'fn save_checkpoint(&self, dir: &Path, step: u32, loss: f32, rank: u32, + alpha: f32) -> Result' + status: implemented + notes: Serializes LoRA A/B + optimizer state to JSON +- contract: wgpu-transformer-trainer-v1.yaml + equation: C-WGPU-CKPT-002 + module_path: entrenar::train::transformer_trainer::wgpu_trainer + function: load_checkpoint + signature: 'fn load_checkpoint(&mut self, path: &Path) -> Result<(u32, f32), String>' + status: implemented + notes: Restores LoRA adapters with dimension validation +critical_path: +- forward +- checkpoint +- loss +- gradient +- AprWriter +- CudaTransformerBlock diff --git a/contracts/lint-baseline.json b/contracts/lint-baseline.json index 3dc3abed58..ea83b36ef8 100644 --- a/contracts/lint-baseline.json +++ b/contracts/lint-baseline.json @@ -1,17 +1,13 @@ { - "_spec": "APR-RELEASE-001 §11.2 — moves only through `make ont-ratchet` (ONT R-6)", + "_spec": "APR-RELEASE-001 §11.2 counters are MEASURED comparand->head by scripts/check_ont_ratchet.sh (#3569); this file holds decisions only", "armed_gates": ["validate", "audit", "score", "verify", "enforce", "enforcement-level", "duplicate-stems", "composition", "sigma", "relations", "shapes"], - "armed_shapes": ["ont-shapes-v1", "ladder-measured", "ladder-green"], + "armed_shapes": ["ont-shapes-v1", "ladder-measured", "ladder-green", "capability-cells"], + "contracts_without_valid_under": 386, + "command": "make lint-ratchet", + "unpaired_theorem_modules": 130, + "contracts_without_depends_on": 278, "ont": { - "consumer_present": true, - "contracts_total": 1882, - "entity_types_registered": 11, - "extractors_implemented": 8, - "contracts_anchored": 7, - "contracts_shaped": 6, - "unanchored_but_bindable": 297, "formal_prose": 1464, - "legacy_unresolved_depends_on": 8, - "shapes_unarmed": 15 + "legacy_unresolved_depends_on": 8 } } diff --git a/contracts/model-capability-ladder-v1.yaml b/contracts/model-capability-ladder-v1.yaml index 8c6859e96e..a0901c96cf 100644 --- a/contracts/model-capability-ladder-v1.yaml +++ b/contracts/model-capability-ladder-v1.yaml @@ -58,9 +58,169 @@ ladder: hosts: - { id: lambda, isa: x86_64, gpu: "RTX 4090", cc: sm_89, required: true } - { id: gx10, isa: aarch64, gpu: "GB10", cc: sm_121, required: true } + # #3943 (b), cop ruling 2026-09-23: the `serve` probe waits on the SERVER, not a clock. + # It waits while the process is alive AND making progress (serve log growing, or the + # process tree's CPU time growing). A load quiet on both for stall_s is HUNG -> RED. + # ceiling_s is the one hard bound on the whole wait, for every rung. A process that + # dies mid-load is RED at once. The old fixed 90 s was shorter than a 27B load on + # gx10 (the server reached "listening" after the probe had given up), so stall_s keeps + # the old number as a window of NO PROGRESS instead of a total. + serve_health: + stall_s: 90 + ceiling_s: 900 + # #3710 ruling 3 / #4022 (operator 2026-09-23, relayed by aprender-cf; "recommended for all questions"): + # the release commit X = 4f81bdea7 + #4022, a scoped hotfix. A receipt measured at `receipts_at` + # binds X ONLY when every path of the diff matches `paths` AND Cargo.lock changes no dependency + # (scripts/lib/ladder_equiv.py). The gate prints the diff it accepted; anything outside is STALE BY SHA. + # #4051 (#4033 lever 0): from this version on, every measured row carries a stamp per apr call + # (t_start / t_acquired / t_end, UTC s; lock_wait_s; lock gpu|none) for qa, code, and run/chat/serve on + # each backend. check_model_ladder.sh refuses a row missing one, or with t_end < t_start, by name. + timing: + required_from: 0.70.0 + ticket: '#4051' + hotfix_scope: + receipts_at: 4f81bdea7bdc17da7a4b22823ef000671b78a435 + paths: ["crates/aprender-mcp/**", "crates/apr-cli/Cargo.toml", "Cargo.lock", "docs/**", + # #4032 (X2 = 1eb1fe808 + one commit, aprender-6c [3ada9a]): its two files EXACTLY -- a + # Qwen3.5-0.8B load-time known-issue warning. No lock or manifest change. Cop ruling 2026-09-23. + "crates/aprender-serve/src/gguf/inference/forward/forward_qwen35.rs", + "crates/aprender-serve/src/gguf/inference/forward/qwen35_known_issue.rs"] + ruling: "#3710 ruling 3 / #4022 + #4032" + # OPERATOR OVERRIDE, 2026-09-23, relayed verbatim by the release cop aprender-cf: "nope, override. I am + # the boss". #4022 (X = 1eb1fe808) removes aprender-mcp's `anyhow` edge -- a dependency change that + # ruling 3 excludes. The override is pinned to EXACTLY that Cargo.lock delta: its canonical JSON + # (ladder_equiv.lock_delta) hashes to delta_sha256 below. Any other lock change hashes differently + # and stays RED; the gate prints "OPERATOR OVERRIDE (#4022 lock delta)", never plain green. + lock_overrides: + - delta_sha256: d387a7e50c91b0f13ec1c924c06b9dcc46716cebf935357a8b9c6c0f9ee8f079 + delta: '{"changed":{"aprender-mcp":{"attrs":null,"edges_added":[],"edges_removed":["anyhow"]}},"packages_added":[],"packages_removed":[]}' + ticket: "#4022" + date: "2026-09-23" + ruling: '"nope, override. I am the boss" (operator, verbatim, via aprender-cf)' + # OPERATOR EMERGENCY SCOPE for 0.69.1 ONLY (relayed verbatim by the release cop aprender-cf, + # 2026-09-23). `check_model_ladder.sh --scope crux-smoke` judges CRUX smoke receipts from the RELEASE + # BINARY instead of the ladder (scripts/lib/crux_smoke_scope.py): every host below, every certified + # model (from the prompt certification), every thinking mode below, all GREEN with a GREEN positive + # control, receipts bound to the cut exactly. Refused for any other release. + # #4045: the NORMAL release gate from 0.70.0 (check_model_ladder.sh --scope release --nightly ). Operator, + # relayed by the release cop aprender-cf, 2026-09-23: "lets update our process for .70 and one of your workers on + # that epic ensure new optimization is part of workflow and we change nightly for the long releaes"; and "anything + # huge, must be nightly only". A release = CRUX smoke on the RELEASE BINARY (these hosts x the certification's + # admitted modes, positive control GREEN) AND a GREEN nightly long certification <= 24 h old at an ancestor, + # bound to the cut by equivalence / the #4037 carry. The emergency scope below is 0.69.1's alone. + release_gate: + from: "0.70.0" + ruling: '#4045, operator 2026-09-23: "we change nightly for the long releases" / "anything huge, must be nightly only"' + crux_smoke: + hosts: [lambda, gx10] + thinking: ["off", "on"] + nightly: + max_age_h: 24 + emergency_scopes: + - name: crux-smoke + release: "0.69.1" + date: "2026-09-23" + quote: '"for this particular release we need emergency surgery. after a binary is build we really only care about CRUX on gx10 and lambda (but "smoke test") style. anything huge, must be nightly only"' + hosts: [lambda, gx10] + thinking: ["off", "on"] + # KNOWN REDs that ship with their ticket (operator selection via the release cop aprender-cf, + # 2026-09-23: "Ship as known RED + ticket"; recorded on #3907). check_model_ladder.sh covers a row + # only when EVERY reason it fails matches a clause pinned to this rung + sha256; anything else on it, + # or this clause on any other model, is RED. An entry that covers nothing on a sweep is STALE and + # RED, so it retires itself when #4030 lands. + known_red: + - rung: qwen35-0.8b-q4km + sha256: bd258782e35f7f458f8aced1adc053e6e92e89bc735ba3be89d38a06121dc517 + clause: + - '^golden_output FAIL: golden_output_thinking_on: think block unclosed within 2048 tokens\b' + - '^apr qa exited \d+ — gates_failed: golden_output \(#3898\)$' + ticket: "#4030" + ruling: "operator selection via aprender-cf 2026-09-23 on #3907: ship as known RED + ticket" inventory: + # #3957 F2: `apr_sha_drift` (#3940) is DELETED. It declared, in prose no code read, that + # receipts measured at 9b7739951 stood for the cut 28c24207a. check_model_ladder.sh now + # binds every receipt to the cut by `apr_sha` (equal, or equal-tree outside evidence/) + # and refuses this key by name, so a delta is re-measured, never declared. dirs: ["~/models", "~/.apr/models", "~/.cache/apr/models"] # depth 1, the fleet's model dirs - patterns: ["*q4_k*.gguf", "*q4k*.gguf", "*q4_k*.apr", "*q4k*.apr"] # case-insensitive + # WIDENED 2026-09-22 by operator ruling: "don't leave behind. if it fits on this machine + # and/or gx10 with CUDA we must support for Qwen 3.5 and 2.5. we kick the MoE work to later". + # MEASURED before the change: 28 models held on lambda, 18 swept, 10 INVISIBLE TO EVERY GATE, + # because the universe was a filename glob over q4_k/q4k only. That is #3712's own defect in a + # new form -- it replaced a hand-picked list with a glob, which has the same omission property, + # moved from a human decision to a string match nobody reviews (#3846). Escapees included + # Qwen3-Coder-30B-A3B-Instruct-Q4_0 (a Q4 quant `*q4_k*` does not match) and + # Qwen3.5-35B-A3B-UD-IQ4_XS (a 35B Qwen3.5 no gate had ever loaded). The two `.apr` files + # escaped in PRINCIPLE rather than by accident: an .apr filename need not carry its + # quantisation at all, so a name-based universe cannot see them. The durable fix is to derive + # the universe from the file HEADER, which `apr inspect --json` already reports (#3846); these + # four patterns close the 2.5/3.5 hole now. Adds 7 measurable models (0.34-7.11 GB: IQ2_XXS, + # IQ3_M, IQ4_XS x2, f16, and both .apr) plus one deferred MoE. + patterns: ["*q4_k*.gguf", "*q4k*.gguf", "*q4_k*.apr", "*q4k*.apr", + "*qwen2.5*.gguf", "*qwen3.5*.gguf", "*qwen2.5*.apr", "*qwen3.5*.apr"] # case-insensitive + # DEFERRED, per the same ruling's "we kick the MoE work to later". A match is printed DEFERRED + # and NEVER counted green: this build has no CUDA forward for qwen3moe at all (#3714), so an + # A3B model owes cells no code can satisfy. This is a DECLARED deferral carrying a reason, not + # an exemption -- an inventory model that fails and is not named here is still refused, which + # the case table proves. Delete this key when #3714 lands. + # EMPTY. The last deferral, "*A3B*", is RETIRED with this fold: the qwen3moe CUDA forward + # (#3714 R2, PMAT-3714-r2-on-v2@6f6fa9a83) is folded into 0.69.1 on the operator ruling of + # 2026-09-23, relayed verbatim by aprender-6a (cop): "fold in MoE". An A3B model now owes + # every cell like any other rung, and #3817 (run --gpu silently falling back to CPU) must be + # shown fixed by the envelope, not assumed from parity. + deferred: {} + # THE THREE FILENAME DEFERRALS DECLARED HERE ON 2026-09-23 ARE RETIRED. + # Operator doctrine, same day, verbatim: "no defer; from now forward we practice + # toyota way and oracle approach of crux with capabiltiy report style from Alfredo". + # A capability is ORACLE-PROVEN or it is RED; there is no third state, so + # "the model runs but tensor_contract cannot inspect it" is RED and the work is the + # missing IQ dequantizer in aprender-core (#3850), not a declaration. Stop the line. + # retired: Qwen2.5-0.5B-Instruct-IQ3_M.gguf + # Qwen2.5-0.5B-Instruct-IQ4_XS.gguf + # Qwen3.5-4B-UD-Q4_K_XL.gguf + # *A3B* was the one deferral kept past that doctrine, as an explicit operator ruling of + # 2026-09-22 ("we kick the MoE work to later"). It is retired by the ruling of 2026-09-23 + # above, which makes #3714 a 0.69.1 requirement. + # #3957 F9/F10: NAMED RED verdicts. Unlike `deferred`, they are not declarations. Each one is + # a claim about CAUSE that check_model_ladder.sh admits only when THIS sweep re-proves it + # (scripts/lib/model_ladder_redmodel.py); otherwise the row is plain FAIL and blocks. A proven + # cell prints RED-MODEL / RED-UNSUPPORTED, is never counted green, and does not block. A key + # matching no held file, or matching a green row, is refused. + # + # F9, operator ruling 2026-09-23 on #3957 (verbatim): "proceed with (a)". The FILE is + # defective, as proven each sweep by llama.cpp on the identical GGUF (the CRUX greedy rows), + # a positive control on the sibling, and apr CPU == GPU. Covers thinking ON only. + # red_model: EMPTY. Both candidates were REFUTED by the F9 oracle itself (aprender-83, + # evidence/crux/0.69.1/f9-greedy-apr-70d20d44c on PMAT-3952-crux-greedy2@a78301558): on the + # model's OFFICIAL thinking-ON template, llama.cpp closes the think block and answers + # correctly for Qwen3.5-0.8B-IQ4_XS (332 tok) and 0.8B-Q4_K_M (241 tok). The "never closes" / + # "closes empty" behaviour came from apr's non-official ON prompt (#3990), in BOTH engines. + # That is an apr defect, and a key for it would have laundered one into RED-MODEL. + # + # wrong_answer (cop ruling 2026-09-23, within ruling (a)): Qwen3.5-0.8B-UD-IQ2_XXS answers + # "What is 2+2?" without "4" on the OFFICIAL template, in apr AND llama.cpp (aprender-6c + # [3ada9a], #4004), while its Q4_K_M sibling answers "4". The key is INERT until a sweep + # re-proves it, all on the official ids: apr's first divergence from the llama.cpp CPU reference + # comes NO EARLIER than the reference's own CUDA leg diverges from it (parity calibrated per + # sweep against the oracle's noise; 6c measured oracle step 2, apr step 4); apr and both llama.cpp + # backends wrong; the control right in both engines; apr CPU == GPU. Until then the row is plain + # RED, as it would be without the key. + red_model: + Qwen3.5-0.8B-UD-IQ2_XXS.gguf: + ticket: "#4004" + defect: wrong_answer + thinking: "off" + expect: "4" + # the certified v2 2+2 control the sweep runs (dd's set; `golden-2plus2` is v1, not in it -- + # aprender-83). The claim is re-proven on the prompt the sweep measures, never on the v1 text. + prompts: [ctl-2plus2] + control: Qwen3.5-0.8B-Q4_K_M.gguf + # F10, operator ruling 2026-09-23 on #3957 (verbatim): "c". apr has no CUDA forward for + # qwen35moe (hybrid SSM + MoE) and must REFUSE it by name on `apr run --gpu`: no fallback, no + # output (aprender-serve capability::no_cuda_forward_reason). The CUDA forward is 0.70.0 work. + red_unsupported: + "Qwen3.5-35B-A3B*.gguf": + architecture: qwen35moe + ticket: "#3977" backends: [cuda] # #3712 row B (operator 2026-09-21: "MORE than enough tokens for our needs (overdo it), and our verbs are # dogfooded, i.e chat with and without thinking, ditto run, ditto code"). A cell is (model, host, verb, @@ -73,12 +233,27 @@ ladder: # the families below plus ONE representative per other arch, named here by file (cop 2026-09-21). cells: verbs: [run, chat, serve, code] + # #3828: a removal from this block must be DECLARED here with a reason, or the + # anti-shrink floor in check_model_ladder.sh refuses it as a silent shrink. The + # declaration is printed on every run, so a withdrawal cannot decay into an absence + # nobody re-reads. This is the mechanism #3817 needed and did not have. + withdrawn: + representatives: + qwen3moe: >- + #3817, 0.69.1: not a fit question and not a regression — this build has NO CUDA + forward for the qwen3moe architecture at all (#3714, lands 0.70.0). `apr run --gpu` + now refuses it by name at pre-load rather than silently falling back to CPU, so a + long-rung representative for it would owe cells no code can satisfy and a + RefusalCell would be a false arithmetic claim. Restore this key with #3714. long_rungs_for: families: [qwen35] # every consumer's family: the full rung set, through declared representatives: # one file per other arch the fleet holds qwen2: qwen2.5-coder-7b-instruct-q4_k_m.gguf # lambda + gx10 qwen3: Qwen3-8B-Q4_K_M.gguf # lambda + gx10 - qwen3moe: Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf # gx10 (128 GB unified); fits no 24 GB card long + # qwen3moe: REMOVED for 0.69.1 (#3817). Not a fit question — this build has no CUDA + # forward for the architecture at all (#3714, lands 0.70.0), so `apr run --gpu` refuses + # pre-load by name and there is no GPU cell to owe. The CPU path is unchanged and works. + # Restore this line with the GPU forward, not before. rungs: - id: qwen2-1.5b-q4km family: qwen2.5 @@ -102,8 +277,11 @@ ladder: gguf: Qwen3-8B-Q4_K_M.gguf sha256: d98cdcbd03e17ce47681435b5150e34c1417f50b5c0019dd560e4882c5745785 backends: [cpu, cuda] - required: false # REFUSED by check_model_ladder.sh (#3712): T-2 is RED while this key stands - note: "second dense-Qwen3 size. No Q4_K model is optional (#3712), so the release gate refuses this key and the release cannot ship on it. It is not flipped here because the ARMED ladder-green shape reads the committed 0.68.2 lambda receipt (golden_output: Empty output — the 0.69.0 hold) and would turn pv lint red on every PR; the flip to true lands with the Qwen3-8B CUDA fix and a committed green lambda receipt" + required: true # flipped 2026-09-22 (#3724): the empty-output defect this key was hiding is FIXED + # and MEASURED green 8/8 on BOTH lambda (where it failed) and gx10 at 3ebb1c61e. + # `required: false` on a Q4_K rung is what let 0.69.0 ship it as a note instead of a + # failure -- FALSIFY-MCL-011's own if_fails says so -- and no Q4_K model is optional. + note: "second dense-Qwen3 size. No Q4_K model is optional (#3712), so the release gate refuses this key and the release cannot ship on it. FLIPPED to true for 0.69.1: the condition this note used to name as the reason for deferring — 'the flip lands with the Qwen3-8B CUDA fix and a committed green lambda receipt' — is the release's own work, not a reason to wait. Arming it surfaced what the key had masked since 0.69.0: the committed 0.68.2 lambda receipt carries qa_rc 5 / golden_output 'Empty output' / green false, so ladder-green fires missingGreenHost: lambda and takes contracts, pv lint, check_model_ladder and clean-room red together — ONE cause, four gates. resolve() unions green witnesses ACROSS versions, so a green 0.69.1 lambda receipt closes all four and no committed receipt is edited" - id: qwen35-0.8b-q4km family: qwen3.5 arch: qwen35 diff --git a/contracts/ont-capability-cells-v1.yaml b/contracts/ont-capability-cells-v1.yaml new file mode 100644 index 0000000000..22cd111ef6 --- /dev/null +++ b/contracts/ont-capability-cells-v1.yaml @@ -0,0 +1,156 @@ +# ───────────────────────────────────────────────────────────────────────────── +# ont-capability-cells-v1 — a required cell of the model-capability ladder is oracle-proven or RED +# (ONT-001 v4.12 §5 ONT-4c5, paiml/infra 948ae923; aprender #3972 + #4047, PMAT-3972) +# +# WHAT IT GOVERNS +# provable_contracts::ontology::capability_cells — D, the cells measured at the current release V*, and +# not_run = D \ {Pass | Fail}; one `apply` for corpus and plant. +# provable_contracts::lint::capability_cells_gate — the validator's half inside `pv lint --gate shapes`: the +# positive control through the ARMED shape, the run-time +# wiring check, a finding naming every NotRun cell, and +# |D| = 0 as a decline. +# +# THE RULE. The domain D is the set of cells ONT-4c1's `ladder-green` quantifies over (receipts::expected_hosts, +# one definition), restricted to `required: true` rungs; a cell id is `@`. A cell is MEASURED only by +# a witness row (sha256 = the rung's) in a receipt of the current release V* (the greatest dotted-numeral +# `version` read) for that host. Its verdict is Pass (ladder-green's row predicate), Fail (its negation), or +# NotRun: a row label that `Verdict::from_label` maps to Unknown(_) — DEFER, MANUAL, NO-VERDICT, … — beats the +# measurement, and a label nobody knows is refused by name. Every D member that is not Pass or Fail is a +# `model:notRunCell`, and the ARMED shape says maxCount 0 on it: Fail, exit 1 — the arming rule. An absent cell is +# NEVER folded into Fail: this row admits Fail (ladder-green refuses it), so a folded absence would pass. +# +# WHY V*, NOT ladder-green's cross-version "∃ witness". Every required rung carries one sha across 0.68.1, +# 0.68.2 and 0.69.1; unioning versions would let a 0.68.2 green row hide a 0.69.1 DEFER or a missing 0.69.1 row +# (plan grill round 1, measured on this tree). ladder-green itself is unchanged. +# +# THE PLANT. A required rung with no receipt (tests/fixtures/ont/capability-cells-plant.yaml, embedded +# byte-identically in the crate) runs every gate run through the same `apply`, then through the ARMED shape; both +# must refuse exactly `capability-cells-plant@plant-host`, or pc_shapes["capability-cells"] is not-fired and the +# gate declines. The corpus wiring is checked at run time too: the rungs the shape flags must be the rungs owning +# a NotRun cell, else the gate declines (Differential). +# +# RESIDUAL, NAMED (the row's own). D takes ONT-4c1's host axis unchanged: `hosts:` absent = every host with a +# receipt read. A host that has never had a receipt is not in D; the probe's literal baseline set B guards the +# 14 named cells. +# +# KIND: pattern. Shapes over a graph; the proof is the gate's own case table. +# ───────────────────────────────────────────────────────────────────────────── +name: ont-capability-cells +version: "1.0.0" +scope: > + Which cells of the model-capability ladder a release owes a verdict on (D), how a cell is measured at the + current release, and the armed shape that turns a required cell without a verdict into RED. Out of scope: + whether a Fail cell may ship (ladder-green, ONT-4c1), per-verb/thinking cells (the v2 cells[] axis, #3712), + the oracle differential (ONT-4b2). +status: active + +metadata: + version: "1.0.0" + kind: pattern + created: '2026-09-23' + last_modified: '2026-09-23' + author: PAIML Engineering + description: > + A required ladder cell is Pass or Fail at the current release V*; NotRun (missing, DEFER, MANUAL, + NO-VERDICT, an unknown label) is a violation of the armed capability-cells shape, exit 1; |D| = 0 declines. + references: + - 'paiml/infra docs/specifications/paiml-ontology.md v4.12 §5 ONT-4c5 (948ae923, infra#950); docs/audits/ont-4c5-plan.md there' + - 'aprender #3972 (extractor + arming rule), #4047 (pv lint capability-cells shape); docs/audits/impl-PMAT-3972-plan.md' + - 'crates/aprender-contracts/src/ontology/{capability_cells.rs, receipts.rs, verdict.rs}, lint/{capability_cells_gate.rs, shapes_gate.rs}' + +entity: + type: gguf + +relations: + depends_on: [ont-shapes-v1, ont-model-receipts-v1, model-capability-ladder-v1] + +shapes: + - id: capability-cells + targetClass: model:RequiredModel + properties: + - {path: model:notRunCell, maxCount: 0} + - {path: model:refusedLabel, maxCount: 0} + +equations: + domain: + formula: "D = { rung@host : rung.required ∧ host ∈ expected_hosts(rung, receipt_hosts(R)) }" + domain: "every rung of the ladder contract; R = every receipt read under evidence/dogfood/models/" + codomain: "a set of cell ids @" + invariants: + - "expected_hosts is ONT-4c1's function, called, not copied: D and ladder-green quantify over one host axis" + - "a rung with required: false contributes no cell" + preconditions: + - "every receipt version is dotted numerals, else the gate refuses by name (exit 3)" + postconditions: + - "|D| = 0 ⇒ the gate declines (exit 2), never Pass" + lean_theorem: none — L4 not declared + not_run: + formula: "not_run = D \\ { c ∈ D : cell(c, V*) ∈ {Pass, Fail} }, V* = max(version(R))" + domain: "D and the witness rows of the V* receipts" + codomain: "a model:notRunCell literal per member, on the rung's model:Model node" + invariants: + - "a row label mapping to Unknown(_) beats the measurement; an unknown label is refused by name and NotRun" + - "several V* rows for one cell combine NotRun > Fail > Pass — not the lattice meet" + - "a row of an older version never measures a cell" + preconditions: + - "a witness row is one whose sha256 equals the rung's; no sha256 or another hex is no measurement" + postconditions: + - "|not_run| > 0 ⇒ the armed capability-cells shape fails ⇒ exit 1" + lean_theorem: none — L4 not declared + +invariants: + - id: CAP-INV-001 + property: a required cell without a verdict fails the armed shape + formal: 'c ∈ D ∧ cell(c, V*) ∉ {Pass, Fail} ⇒ Fail(capability-cells)' + prose: false + - id: CAP-INV-002 + property: an absent cell is never folded into Fail + formal: 'c ∈ D ∧ witness(c, V*) = ∅ ⇒ c ∈ not_run' + prose: false + - id: CAP-INV-003 + property: an older version never masks the current release + formal: 'version(r) < V* ⇒ r ∉ witnesses(c) for every c ∈ D' + prose: false + - id: CAP-INV-004 + property: an empty domain declines + formal: '|D| = 0 ⇒ verdict = Unknown' + prose: false + - id: CAP-INV-005 + property: the plant fires through the armed shape or the gate declines + formal: 'pc_shapes(capability-cells) ≠ fired ⇒ verdict = Unknown' + prose: false + +falsification_tests: + - id: FALSIFY-CAP-001 + rule: cells at V*, label precedence, duplicate rows + prediction: "a missing cell and a DEFER/MANUAL/NO-VERDICT row are not_run; an older green row does not mask either; a stray higher version only adds NotRun; a non-numeral version is refused by name; a green duplicate does not hide a DEFER or a Fail; a Fail cell is admitted" + test: cargo test -p aprender-contracts --lib ontology::capability_cells + if_fails: a required cell without a verdict arms, or a Fail is refused by the wrong shape + - id: FALSIFY-CAP-002 + rule: the plant is the tracked fixture and fires through the shape + prediction: the embedded plant equals tests/fixtures/ont/capability-cells-plant.yaml byte for byte, and refuses exactly capability-cells-plant@plant-host through the armed shape + test: cargo test -p aprender-contracts --lib lint::capability_cells_gate + if_fails: the positive control drifts from the tracked one, or stays fired with the shape gone + - id: FALSIFY-CAP-003 + rule: the gate's answers on the CLI + prediction: "Pass on the real corpus with capability-cells armed, pc_shapes fired, not_run empty and B ⊆ domain; exit 1 naming the cell on a missing, DEFER, MANUAL or NO-VERDICT required cell; exit 2 decline when D is empty or capability-cells is disarmed; a Fail cell exits 0 under this shape" + test: cargo test -p aprender-contracts-cli --test ont4c5_capability_cells + if_fails: a decline, a reject and a pass become indistinguishable to the probe + - id: FALSIFY-CAP-004 + rule: the spec row's mutations + prediction: "fold a missing cell into Fail → the plant passes → decline; drop capability-cells from armed_shapes → decline; flip one required rung to required: false → B ⊄ D → the probe is RED; label a required cell DEFER → exit 1 naming it" + test: cargo test -p aprender-contracts-cli --test ont4c5_capability_cells + if_fails: the row's own probe cannot go RED on the mutations it names + +qa_gate: + id: F-CAP-001 + name: capability-cells + description: 'A required cell of the model-capability ladder is Pass or Fail at the current release; NotRun is RED (ONT-4c5)' + checks: + - domain_is_ladder_greens + - cells_measured_at_v_star + - not_run_never_admitted + - empty_domain_declines + - plant_fires_through_armed_shape + pass_criteria: All 4 falsification tests pass; `pv lint contracts/ --gate shapes --format json` is Pass on the real corpus with capability-cells in armed_shapes, pc_shapes["capability-cells"] fired, capability_cells.not_run empty and the 14 ids of B in capability_cells.domain + falsification: 'label one required 0.69.1 row DEFER → exit 1 naming @; delete one required row from a 0.69.1 receipt → exit 1 naming it' diff --git a/contracts/ont-code-symbols-v1.yaml b/contracts/ont-code-symbols-v1.yaml index 2240e234e2..959f83f03b 100644 --- a/contracts/ont-code-symbols-v1.yaml +++ b/contracts/ont-code-symbols-v1.yaml @@ -45,7 +45,7 @@ metadata: extract:code — the bound symbols by a syn module-tree walk (inline mods, file mods, #[path], one re-export hop, impl methods), fail-closed with the reason in the graph; extract:lean — theorems from the in-tree Theorems tree, sorry-free files only grounding, joined to the contracts that cite them by ONT-2a's name - forms; 16 vendored W3C SHACL-Core cases run every gate run; the oracle differential in the release gate. + forms; 19 vendored W3C SHACL-Core cases run every gate run; the oracle differential in the release gate. references: - 'paiml/infra docs/specifications/paiml-ontology.md §3.6 (the subset), §3.7 (extractors), §5 ONT-4b2, R-13, R-15, R-18' - 'paiml/infra docs/specifications/ont-oracle-ledger.md (ONT-0: the 32-case table and the pinned oracle)' diff --git a/contracts/ont-valid-under-v1.yaml b/contracts/ont-valid-under-v1.yaml new file mode 100644 index 0000000000..b8b2927470 --- /dev/null +++ b/contracts/ont-valid-under-v1.yaml @@ -0,0 +1,119 @@ +# ───────────────────────────────────────────────────────────────────────────── +# ont-valid-under-v1 — kernel-kind contracts carry a world index (ONT-7) +# (ONT-001 §4.2 `metadata.valid_under{}`, §5 ONT-7; issue #4076; ticket PMAT-4076) +# +# WHAT IT GOVERNS +# provable_contracts::lint::valid_under_gate — PV-ONT-013..016, the census, +# the `contracts_without_valid_under` ratchet; `pv lint --gate valid-under` +# and gate 13 of the full run (R-8: computed everywhere, armed per repo). +# +# THE SCHEMA IS THIS ROW'S DESIGN. The spec names the key and gives one +# example (Appendix B: qualifiers, no `world`); it pins no schema. `world` +# indexes Σ's `worlds:`; omitted, it reads `committed`, which Σ documents as +# "the world every contract is read in unless it says otherwise" — so the +# spec's own example is admitted as written. +# ───────────────────────────────────────────────────────────────────────────── +name: ont-valid-under +version: "1.0.0" +scope: > + Where a kernel-kind contract's claim holds: the world it names in Σ, and the toolchain, host class, backend and + feature set that qualify it. Out of scope: whether a qualifier's VALUE is a real toolchain or backend (no + vocabulary is declared for them yet), and what a world means to pv-sat (ONT-5). +status: active + +metadata: + version: "1.0.0" + kind: pattern + created: '2026-09-24' + last_modified: '2026-09-24' + author: PAIML Engineering + description: > + A contract may say where its claim holds with `metadata.valid_under`; the valid-under gate refuses a block that + is not a mapping, is empty, carries an undeclared key, names a world Σ does not declare, or has a malformed + qualifier, and refuses a rise in the kernel-kind contracts that carry none. + references: + - 'paiml/infra docs/specifications/paiml-ontology.md §4.2 (metadata.valid_under), §5 ONT-7, Appendix B, R-2, R-8' + - 'crates/aprender-contracts/src/lint/valid_under_gate.rs — PV-ONT-013..016' + - 'contracts/ontology.yaml — worlds (committed), readers ontology/sigma.rs and lint/valid_under_gate.rs' + +relations: + depends_on: [ont-sigma-v1, ont-verdict-lattice-v1] + +equations: + valid_under_admission: + formula: "admits(corpus) ⇔ ∀ c ∈ corpus: valid_under(c) = ∅ ∨ (keys(valid_under(c)) ⊆ K ∧ world(c) ∈ Σ.worlds ∧ wellformed(c)) ∧ without(corpus) ≤ baseline" + domain: "a contract corpus and the Σ that declares its worlds" + codomain: "Pass | Fail | Unknown" + invariants: + - "a corpus whose every valid_under resolves its world and whose debt did not rise is admitted" + - "a corpus with no kernel-kind contract and no valid_under is Unknown, never Pass" + preconditions: + - "Σ parses and satisfies its own integrity rules" + postconditions: + - "verdict ∈ {Pass, Fail, Unknown}" + - "contracts_without_valid_under ≤ baseline(contracts_without_valid_under)" + lean_theorem: none — L4 not declared + +invariants: + - id: VU-INV-001 + property: a valid_under block carries only the five declared keys + formal: 'k ∈ keys(valid_under) ⇒ k ∈ K' + prose: false + - id: VU-INV-002 + property: the world index resolves into Σ, an omitted world reading committed + formal: 'world(c) ∈ Σ.worlds' + prose: false + - id: VU-INV-003 + property: a valid_under block is never empty + formal: 'valid_under(c) ≠ ⊥ ⇒ len(keys(valid_under(c))) ≥ 1' + prose: false + - id: VU-INV-004 + property: the kernel-kind debt is shrink-only + formal: 'without(corpus) ≤ baseline' + prose: false + +falsification_tests: + - id: FALSIFY-VU-001 + rule: every rule fires on its own fixture, and only its own + prediction: the nine valid-under fixtures draw exactly PV-ONT-013/014/015/016 as tabled, and the two well-formed ones pass + test: cargo test -p aprender-contracts --lib lint::valid_under_gate::tests::every_fixture_draws_exactly_its_rule + if_fails: a rule is dead, or fires on the wrong input + - id: FALSIFY-VU-002 + rule: the spec's own example is admitted + prediction: Appendix B's qualifiers with no world pass and are counted as committed + test: cargo test -p aprender-contracts --lib lint::valid_under_gate::tests::an_omitted_world_reads_committed_so_appendix_b_is_admitted + if_fails: the gate refuses the instance the spec gives as the shape of the key + - id: FALSIFY-VU-003 + rule: an undeclared world + prediction: "world: mars is PV-ONT-014 naming the worlds Σ declares" + test: cargo test -p aprender-contracts --lib lint::valid_under_gate::tests::an_undeclared_world_is_pv_ont_014 + if_fails: the world index indexes nothing + - id: FALSIFY-VU-004 + rule: the debt ratchet + prediction: a rise above the baseline is PV-ONT-016; equal or lower passes; no baseline is reported, never judged + test: cargo test -p aprender-contracts --lib lint::valid_under_gate::tests::the_ratchet_rejects_a_rise_only + if_fails: a kernel contract with no world can be added unrecorded + - id: FALSIFY-VU-005 + rule: zero is a decline, a bad block is still a reject + prediction: "no kernel and no valid_under declines; a bad valid_under on a non-kernel contract with no kernel in sight rejects" + test: cargo test -p aprender-contracts-cli --test ont7_valid_under_gate + if_fails: a decline and a reject become indistinguishable to a caller + - id: FALSIFY-VU-006 + rule: R-8 + prediction: the full `pv lint` run computes valid-under, reported and not armed + test: cargo test -p aprender-contracts --lib lint::tests + if_fails: armed_gates could name a gate no run computes + +qa_gate: + id: F-VU-001 + name: world index + description: A kernel-kind contract's claim names the world it holds in (ONT-7) + checks: + - closed_key_set + - world_resolves_in_sigma + - not_empty + - qualifiers_wellformed + - debt_ratchet + - computed_in_every_run + pass_criteria: All 6 falsification tests pass; `pv lint contracts/ --gate valid-under` is Pass on the real corpus + falsification: 'set `world: mars` in ont-verdict-lattice-v1 → PV-ONT-014 naming `committed`; add a kernel contract with no valid_under → PV-ONT-016 (both measured on fixtures 2026-09-24)' diff --git a/contracts/ont-verdict-lattice-v1.yaml b/contracts/ont-verdict-lattice-v1.yaml index 29cc6f0703..697d0f3bf9 100644 --- a/contracts/ont-verdict-lattice-v1.yaml +++ b/contracts/ont-verdict-lattice-v1.yaml @@ -27,6 +27,10 @@ metadata: last_modified: '2026-09-17' author: PAIML Engineering kind: kernel + # ONT-7: the world this lattice's laws hold in. They are pure functions over an enum, so the one + # declared world, the committed tree, is the whole claim; no backend, host or feature qualifies it. + valid_under: + world: committed description: > ONT-001 verdict lattice — Fail < Unknown(Reason) < Pass with meet = min (Kleene K3 strong conjunction); only Pass arms; per-repo armed gates, monotone against the committed comparand diff --git a/contracts/ontology.yaml b/contracts/ontology.yaml index ae7c0aa0c6..c89a127871 100644 --- a/contracts/ontology.yaml +++ b/contracts/ontology.yaml @@ -147,6 +147,9 @@ entity_types: # aprender#3715: a release's receipts (per-host inventory + cells, kernel diffs, context rungs, the dogfood # receipt R5 judged), extracted ONLY when a release subject is given — an ordinary PR has no release. - {name: release-evidence, extractor: release_evidence, implemented: true} + # aprender#3745 S2 (#3777): the release candidate's own interface (`apr surface --json`, apr-cli-surface/v1), + # extracted ONLY under a release subject with --surface; the release cells are derived from it. + - {name: cli-surface, extractor: cli_surface, implemented: true} extractors: - {name: pv_contract, reader: ontology/extract/pv_contract.rs, implemented: true} @@ -160,6 +163,7 @@ extractors: - {name: csv, reader: ontology/extract/csv.rs, implemented: false} - {name: parity_receipt, reader: ontology/extract/parity_receipt.rs, implemented: true} - {name: release_evidence, reader: ontology/extract/release_evidence.rs, implemented: true} + - {name: cli_surface, reader: ontology/extract/cli_surface.rs, implemented: true} # What this ontology does NOT express, said out loud so nobody encodes it by accident. Each names the reader that # would have to change first. @@ -172,7 +176,7 @@ readers: concepts: ontology/sigma.rs roles: lint/sigma_gate.rs, lint/relations_gate.rs symbols: lint/sigma_symbols.rs - worlds: ontology/sigma.rs + worlds: ontology/sigma.rs, lint/valid_under_gate.rs agents: ontology/sigma.rs entity_types: lint/sigma_gate.rs extractors: ontology/sigma.rs diff --git a/contracts/presentar/binding.yaml b/contracts/presentar/binding.yaml new file mode 100644 index 0000000000..54b976d0cd --- /dev/null +++ b/contracts/presentar/binding.yaml @@ -0,0 +1,221 @@ +version: 1.0.0 +target_crate: presentar +critical_path: +- render +- slide +- export +- compile +bindings: +- contract: dag-ordering-v1.yaml + equation: render + function: render_order + module_path: presentar::crates::presentar-layout::src::compute_block::render_order + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render + module_path: presentar::crates::presentar-terminal::src::compute_block::render + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_frame + module_path: presentar::crates::presentar-terminal::src::tools::bench::render_frame + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_full + module_path: presentar::crates::presentar-terminal::src::direct::diff_renderer::render_full + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_lines + module_path: presentar::crates::presentar-terminal::src::widgets::ux::render_lines + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_string + module_path: presentar::crates::presentar-terminal::src::widgets::micro_heat_bar::render_string + status: implemented +- contract: gpu-context-health-v1.yaml + equation: render + function: render_text + module_path: presentar::crates::presentar-terminal::src::widgets::micro_heat_bar::render_text + status: implemented +- contract: display-format-v1.yaml + equation: render + function: for_render + module_path: presentar::crates::presentar-terminal::src::perf_trace::trackers::for_render + status: implemented +- contract: display-format-v1.yaml + equation: render + function: record_render + module_path: presentar::crates::presentar-terminal::src::app::record_render + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_with + module_path: presentar::crates::presentar-widgets::src::list::render_with + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_range + module_path: presentar::crates::presentar-core::src::virtualization::render_range + status: implemented +- contract: display-format-v1.yaml + equation: render + function: should_render + module_path: presentar::crates::presentar-core::src::virtualization::should_render + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_count + module_path: presentar::crates::presentar-core::src::virtualization::render_count + status: implemented +- contract: display-format-v1.yaml + equation: render + function: cells_to_render + module_path: presentar::crates::presentar-core::src::virtualization::cells_to_render + status: implemented +- contract: display-format-v1.yaml + equation: render + function: should_render_cell + module_path: presentar::crates::presentar-core::src::virtualization::should_render_cell + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_instances + module_path: presentar::crates::presentar::src::webgpu::render_instances + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_json + module_path: presentar::crates::presentar::src::browser::app::render_json + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_counter + module_path: presentar::crates::presentar::src::browser::app::render_counter + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_dashboard + module_path: presentar::crates::presentar::src::browser::app::render_dashboard + status: implemented +- contract: display-format-v1.yaml + equation: render + function: render_widget + module_path: presentar::crates::presentar::src::browser::app::render_widget + status: implemented +# --- TUI rendering contracts (2026-04-02) --- +- contract: tui-rendering-v1.yaml + equation: cellbuffer_bounds + function: get + module_path: presentar::crates::presentar-terminal::src::direct::cell_buffer::get + status: pending +- contract: tui-rendering-v1.yaml + equation: cellbuffer_bounds + function: set + module_path: presentar::crates::presentar-terminal::src::direct::cell_buffer::set + status: pending +- contract: tui-rendering-v1.yaml + equation: diff_renderer_correctness + function: render_diff + module_path: presentar::crates::presentar-terminal::src::direct::diff_renderer::render_diff + status: pending +- contract: tui-rendering-v1.yaml + equation: dirty_tracking + function: mark_dirty + module_path: presentar::crates::presentar-terminal::src::direct::cell_buffer::mark_dirty + status: pending +- contract: tui-rendering-v1.yaml + equation: unicode_width + function: display_width + module_path: presentar::crates::presentar-terminal::src::direct::cell_buffer::display_width + status: pending +- contract: tui-rendering-v1.yaml + equation: color_mode_fallback + function: nearest_256 + module_path: presentar::crates::presentar-terminal::src::color::nearest_256 + status: pending +- contract: tui-rendering-v1.yaml + equation: zero_alloc_render + function: render_diff + module_path: presentar::crates::presentar-terminal::src::direct::diff_renderer::render_diff + status: pending +- contract: tui-rendering-v1.yaml + equation: resize_safety + function: resize + module_path: presentar::crates::presentar-terminal::src::direct::cell_buffer::resize + status: pending +# --- Agent TUI panel contracts (2026-04-02) --- +- contract: tui-panels-v1.yaml + equation: panel_layout_nonoverlap + function: compute_layout + module_path: batuta::src::agent::tui::layout::compute_layout + status: pending +- contract: tui-panels-v1.yaml + equation: adaptive_degradation + function: detail_level + module_path: batuta::src::agent::tui::layout::detail_level + status: pending +- contract: tui-panels-v1.yaml + equation: streaming_token_ordering + function: push_token + module_path: batuta::src::agent::tui::streaming::push_token + status: pending +- contract: tui-panels-v1.yaml + equation: tool_progress_monotonic + function: update_tool + module_path: batuta::src::agent::tui::tools::update_tool + status: pending +- contract: tui-panels-v1.yaml + equation: cost_display_invariants + function: update_cost + module_path: batuta::src::agent::tui::cost::update_cost + status: pending +- contract: tui-panels-v1.yaml + equation: sandbox_violation_visibility + function: add_violation + module_path: batuta::src::agent::tui::sandbox::add_violation + status: pending +- contract: tui-panels-v1.yaml + equation: brick_budget_enforcement + function: verify_all + module_path: batuta::src::agent::brick::verify_all + status: pending +- contract: tui-panels-v1.yaml + equation: statusbar_state_display + function: update_statusbar + module_path: batuta::src::agent::tui::statusbar::update_statusbar + status: pending +# --- TUI lifecycle contracts (2026-04-03) --- +- contract: tui-lifecycle-v1.yaml + equation: widget_lifecycle + function: mount + module_path: presentar::crates::presentar-terminal::src::app::TuiApp::mount + status: pending + notes: Widget lifecycle state machine — Created→Mounted→Rendering→Unmounted +- contract: tui-lifecycle-v1.yaml + equation: widget_lifecycle + function: unmount + module_path: presentar::crates::presentar-terminal::src::app::TuiApp::unmount + status: pending + notes: Unmounted is terminal state — no transitions out +- contract: tui-lifecycle-v1.yaml + equation: render_cycle_correctness + function: render_diff + module_path: presentar::crates::presentar-terminal::src::direct::diff_renderer::render_diff + status: pending + notes: Only dirty cells written to terminal, zero writes for unchanged buffer +- contract: tui-lifecycle-v1.yaml + equation: event_dispatch + function: dispatch_event + module_path: presentar::crates::presentar-terminal::src::app::dispatch_event + status: pending + notes: Key→focused widget, Mouse→hit-test, Resize→re-layout, non-blocking +- contract: tui-lifecycle-v1.yaml + equation: terminal_restore + function: drop + module_path: presentar::crates::presentar-terminal::src::app::TuiApp::drop + status: pending + notes: Terminal restored on all exit paths — normal, panic, signal diff --git a/contracts/pvl-lint-ratchets-v1.yaml b/contracts/pvl-lint-ratchets-v1.yaml new file mode 100644 index 0000000000..e8160db4af --- /dev/null +++ b/contracts/pvl-lint-ratchets-v1.yaml @@ -0,0 +1,121 @@ +# ───────────────────────────────────────────────────────────────────────────── +# pvl-lint-ratchets-v1 — theorem pairing + depends_on, shrink-only (PVL-001 EV-11) +# (paiml/infra PVL-001 @00553b0b §EV-11; issue #4166; ticket PMAT-4166) +# +# WHAT IT GOVERNS +# provable_contracts::lint::ratchet_gates — PV-RAT-001 (theorem-pairing) and +# PV-RAT-002 (depends-on-present); `pv lint --gate theorem-pairing --gate +# depends-on-present` and gates 14/15 of the full run (R-8: computed +# everywhere, armed per repo); `make lint-ratchet` (scripts/lint_ratchet.sh), +# the only writer of the two baselines, which only turns them down. +# +# THE PAIRING RULE IS THIS ROW'S DESIGN. The row says "module name appears in +# book/"; the full dotted module name on identifier boundaries is the reading +# that cannot be satisfied by accident (the book names 80 of 165 file STEMS, +# mostly as ordinary words; measured 2026-09-24). +# ───────────────────────────────────────────────────────────────────────────── +name: pvl-lint-ratchets +version: "1.0.0" +scope: > + Two debts that may fall and never rise: Lean theorem modules no book page names, and kernel-kind contracts + with an empty `metadata.depends_on`. Out of scope: whether a book page's mention is a real explanation, and + whether a declared dependency is the right one. +status: active + +metadata: + version: "1.0.0" + kind: pattern + created: '2026-09-24' + last_modified: '2026-09-24' + author: PAIML Engineering + description: > + `pv lint --gate theorem-pairing` counts Lean theorem modules whose full module name no book page mentions; + `--gate depends-on-present` counts kernel-kind contracts with no depends_on. Each rejects a count above its + top-level baseline in contracts/lint-baseline.json, reports (never passes) with no baseline, declines when + nothing was measured, and never writes the file. + references: + - 'paiml/infra docs/specifications/pvl-001 @00553b0b §EV-11 (probe + mutation)' + - 'crates/aprender-contracts/src/lint/ratchet_gates.rs — PV-RAT-001, PV-RAT-002' + - 'scripts/lint_ratchet.sh — `make lint-ratchet`, the downward-only writer' + +relations: + depends_on: [ont-verdict-lattice-v1] + +equations: + shrink_only_ratchet: + formula: "admits(corpus) ⟺ baseline ≠ ⊥ ∧ debt(corpus) ≤ baseline" + domain: "a contract corpus, its repo root (lean/, book/) and contracts/lint-baseline.json" + codomain: "Pass | Fail | Unknown" + invariants: + - "no baseline is Unknown(Report) with the count printed, never Pass" + - "nothing measured (no Lean base, no module, no book page, no kernel contract) is a decline" + preconditions: + - "the baseline, when present, is a non-negative integer at the top level" + postconditions: + - "verdict ∈ {Pass, Fail, Unknown}" + - "contracts/lint-baseline.json is byte-identical after the gate" + lean_theorem: none — L4 not declared + +invariants: + - id: RAT-INV-001 + property: the unpaired theorem module debt is shrink-only + formal: 'unpaired(corpus) ≤ baseline(unpaired_theorem_modules)' + prose: false + - id: RAT-INV-002 + property: the kernel contracts without depends_on debt is shrink-only + formal: 'without_depends_on(corpus) ≤ baseline(contracts_without_depends_on)' + prose: false + - id: RAT-INV-003 + property: a module is paired only by its full dotted name on identifier boundaries + formal: 'paired(m) ⟺ ∃ p ∈ book: mentions(p, name(m))' + prose: false + - id: RAT-INV-004 + property: a gate never writes its baseline + formal: '¬(after_gate(baseline) ≠ before_gate(baseline))' + prose: false + +falsification_tests: + - id: FALSIFY-RAT-001 + rule: the spec's mutation + prediction: adding an unpaired Theorem module turns `pv lint --gate theorem-pairing --gate depends-on-present` RED with PV-RAT-001 + test: cargo test -p aprender-contracts-cli --test ev11_lint_ratchets + if_fails: a theorem module can be added that no book page names, unrecorded + - id: FALSIFY-RAT-002 + rule: the depends_on ratchet + prediction: a kernel-kind contract with no depends_on above the baseline is PV-RAT-002; equal or lower passes + test: cargo test -p aprender-contracts --lib lint::ratchet_gates::tests::adding_a_kernel_contract_with_no_depends_on_is_red + if_fails: a kernel contract that names no composition can be added unrecorded + - id: FALSIFY-RAT-003 + rule: the pairing boundary + prediction: a file stem, or a longer module name that starts with it, does not pair a module + test: cargo test -p aprender-contracts --lib lint::ratchet_gates::tests::the_file_stem_or_a_longer_name_does_not_pair_a_module + if_fails: the book's ordinary words pair modules by accident + - id: FALSIFY-RAT-004 + rule: no baseline is reported, never judged + prediction: with no baseline key both gates are Unknown(Report), exit 2, the count printed + test: cargo test -p aprender-contracts --lib lint::ratchet_gates::tests::no_baseline_is_reported_and_never_a_pass + if_fails: deleting the baseline turns the ratchet into a pass + - id: FALSIFY-RAT-005 + rule: the gates never write + prediction: the baseline is byte-identical after both gates, on fixtures and on the real corpus + test: cargo test -p aprender-contracts --lib lint::ratchet_gates::tests::neither_gate_writes_the_baseline + if_fails: a gate can raise its own baseline + - id: FALSIFY-RAT-006 + rule: R-8 + prediction: the full `pv lint` run computes theorem-pairing and depends-on-present, reported and not armed + test: cargo test -p aprender-contracts --lib lint::tests + if_fails: armed_gates could name a gate no run computes + +qa_gate: + id: F-RAT-001 + name: lint ratchets + description: Theorem pairing and depends_on debts may fall, never rise (PVL-001 EV-11) + checks: + - unpaired_theorem_modules_shrink_only + - contracts_without_depends_on_shrink_only + - strict_pairing_boundary + - no_baseline_is_report + - gate_never_writes + - computed_in_every_run + pass_criteria: All 6 falsification tests pass; the EV-11 probe exits 0 on the real corpus and `git diff --exit-code contracts/lint-baseline.json` is clean after it + falsification: 'add lean/ProvableContracts/Theorems/S/B.lean named by no book page → exit 1, PV-RAT-001 (measured on a fixture 2026-09-24)' diff --git a/contracts/realizar/binding.yaml b/contracts/realizar/binding.yaml new file mode 100644 index 0000000000..7e1ca8a206 --- /dev/null +++ b/contracts/realizar/binding.yaml @@ -0,0 +1,736 @@ +version: 1.0.0 +target_crate: realizar +bindings: +- contract: forward-pass-v1.yaml + equation: rms_norm + module_path: realizar::gguf::ops + function: rms_norm_unit_offset + status: partial + notes: '#4369: #[contract] site at crates/aprender-serve/src/gguf/ops.rs:128 had no binding row, and contracts/forward-pass-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: gemm-v1.yaml + equation: flattened_batched_gemm + module_path: realizar::gguf::inference::cached::single_flattened_batched_owned + function: flattened_batched_gemm + status: partial + notes: '#4369: #[contract] site at crates/aprender-serve/src/gguf/inference/cached/single_flattened_batched_owned.rs:19 had no binding row, and contracts/gemm-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: gpu-multi-backend-parity-v1.yaml + equation: multi_backend_parity + module_path: realizar::infer::batch_wgpu + function: generate + status: implemented + notes: '#4369: #[contract] site at crates/aprender-serve/src/infer/batch_wgpu.rs:44 had no binding row; row added when the binding moved in-tree' +- contract: gpu-weight-residency-v1.yaml + equation: pcie_overhead + module_path: realizar::infer::batch_wgpu + function: try_init_wgpu_batch + status: implemented + notes: '#4369: #[contract] site at crates/aprender-serve/src/infer/batch_wgpu.rs:129 had no binding row; row added when the binding moved in-tree' +- contract: sampling-v1.yaml + equation: softmax_inplace + module_path: realizar::gguf::ops + function: softmax + status: partial + notes: '#4369: #[contract] site at crates/aprender-serve/src/gguf/ops.rs:347 had no binding row, and contracts/sampling-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: wgpu-forward-pass-v1.yaml + equation: dequant_correctness + module_path: realizar::gpu::adapters::wgpu_adapter + function: dequant_model_weights + status: partial + notes: '#4369: #[contract] site at crates/aprender-serve/src/gpu/adapters/wgpu_adapter.rs:52 had no binding row, and contracts/wgpu-forward-pass-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: matmul-kernel-v1.yaml + equation: quantized_dot + module_path: realizar::quantize::fused_k + function: fused_q4k_dot + signature: 'fn fused_q4k_dot(q4k_data: &[u8], activations: &[f32]) -> Result' + status: implemented + notes: Scalar, AVX2, AVX-512 VNNI variants; also Q6K, Q8_0 +- contract: flash-attention-v1.yaml + equation: flash_attention + module_path: realizar::gguf::inference::attention_part_02 + function: flash_attention_tiled + signature: 'fn flash_attention_tiled(q: &[f32], k: &[f32], v: &[f32], ...) -> Vec' + status: implemented + notes: CPU tiled + CUDA flash attention with causal masking +- contract: attention-kernel-v1.yaml + equation: attention + module_path: realizar::apr::helpers + function: simple_attention + signature: 'fn simple_attention(q: &[f32], k: &[f32], v: &[f32], seq_len: usize, num_heads: usize, num_kv_heads: usize, + head_dim: usize) -> Vec' + status: implemented + notes: Multi-sequence causal attention; GQA-aware with head broadcasting +- contract: gqa-kernel-v1.yaml + equation: gqa + module_path: realizar::apr::helpers + function: simple_attention + signature: 'fn simple_attention(q: &[f32], k: &[f32], v: &[f32], seq_len: usize, num_heads: usize, num_kv_heads: usize, + head_dim: usize) -> Vec' + status: implemented + notes: GQA with head broadcasting (num_heads/num_kv_heads grouping) +- contract: gated-delta-net-v1.yaml + equation: decay + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_decay + signature: 'fn gated_delta_net_decay(a_log: &[f32], dt: &[f32]) -> Vec' + status: implemented + notes: decay = exp(g_h), S *= decay +- contract: gated-delta-net-v1.yaml + equation: read + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_read + signature: 'fn gated_delta_net_read(state: &[f32], q: &[f32]) -> Vec' + status: implemented + notes: 'State read: mem = S^T @ k_norm' +- contract: gated-delta-net-v1.yaml + equation: write + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_write + signature: 'fn gated_delta_net_write(state: &mut [f32], k: &[f32], delta: &[f32], decay: &[f32])' + status: implemented + notes: 'State write: S += k (x) delta' +- contract: gated-delta-net-v1.yaml + equation: delta + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_delta + signature: 'fn gated_delta_net_delta(v: &[f32], read: &[f32], beta: &[f32]) -> Vec' + status: implemented + notes: 'Delta rule: delta = beta * (v - mem)' +- contract: gated-delta-net-v1.yaml + equation: output + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_output + signature: 'fn gated_delta_net_output(y: &[f32], z: &[f32]) -> Vec' + status: implemented + notes: 'Output: o = S^T @ q, then rms_norm_gated' +- contract: rope-extrapolation-v1.yaml + equation: ntk_scaled_base + module_path: realizar::layers::position + function: ScaledRope::new + signature: 'fn new(config: RopeConfig) -> Self' + status: implemented + notes: RopeScalingType::Ntk and DynamicNtk +- contract: rope-extrapolation-v1.yaml + equation: linear_interpolation + module_path: realizar::layers::position + function: ScaledRope::new + signature: 'fn new(config: RopeConfig) -> Self' + status: implemented + notes: RopeScalingType::Linear; context_multiplier tested +- contract: rope-extrapolation-v1.yaml + equation: yarn_ramp + module_path: realizar::layers::position + function: ScaledRope::new + signature: 'fn new(config: RopeConfig) -> Self' + status: implemented + notes: RopeScalingType::Yarn with ramp function +- contract: rope-extrapolation-v1.yaml + equation: yarn_mixed_frequency + module_path: realizar::layers::position + function: ScaledRope::new + signature: 'fn new(config: RopeConfig) -> Self' + status: implemented + notes: YaRN mixed-frequency applied via ScaledRope::apply() +- contract: inference-pipeline-v1.yaml + equation: prefill_phase + module_path: realizar::gguf::inference::forward::batch_part_02_part_03 + function: prefill_batch + signature: 'fn prefill_batch(model: &mut Model, tokens: &[u32], ...) -> Result>' + status: implemented + notes: Batched and chunked prefill; 8.2x speedup over serial +- contract: inference-pipeline-v1.yaml + equation: decode_step + module_path: realizar::gguf::inference::forward + function: Model::forward + signature: 'fn forward(&mut self, token: u32, position: usize) -> Result>' + status: implemented + notes: Single-token forward pass; decode is the default path after prefill +- contract: kv-cache-equivalence-v1.yaml + equation: prefill_incremental + module_path: realizar::gguf::inference::forward::batch_part_02_part_03 + function: prefill_batch + signature: 'fn prefill_batch(model: &mut Model, tokens: &[u32], ...) -> Result>' + status: implemented + notes: KV cache populated incrementally during batched prefill +- contract: kv-cache-equivalence-v1.yaml + equation: batched_serial_equivalence + module_path: realizar::gguf::inference::forward + function: generate_with_batched_prefill + signature: fn generate_with_batched_prefill(...) -> Result> + status: implemented + notes: Batched prefill produces same KV cache as serial +- contract: kv-cache-equivalence-v1.yaml + equation: fused_kernel + module_path: realizar::cuda::executor::kv_cache_part_02_part_03 + function: flash_attention_cached + signature: 'fn flash_attention_cached(q: &CudaSlice, k_cache: &CudaSlice, v_cache: &CudaSlice, ...) -> Result<()>' + status: implemented + notes: CUDA flash attention with fused KV cache read +- contract: qwen35-hybrid-forward-v1.yaml + equation: gdn_sublayer + module_path: realizar::gpu::scheduler::linear_attn + function: gated_delta_net_forward + signature: 'fn gated_delta_net_forward(input: &[f32], weights: &GdnWeights, state: &mut GdnState) -> Vec' + status: implemented + notes: pre-norm -> projections -> causal Conv1D -> SiLU -> gated_delta_rule_step -> RMSNorm_gated -> out_proj -> residual +- contract: qwen35-shapes-v1.yaml + equation: o_projection_transpose + module_path: realizar::gguf::config + function: ArchConstraints::from_gguf_metadata + signature: 'fn from_gguf_metadata(metadata: &GgufMetadata) -> ArchConstraints' + status: implemented + notes: O projection shape [hidden, n_h * d_k] validated by ArchConstraints +- contract: architecture-requirements-v1.yaml + equation: gguf_loader_validation + module_path: realizar::cuda::executor::weights + function: CudaExecutor::build_indexed_weights + signature: 'fn build_indexed_weights(&mut self, num_layers: usize, layer_prefix_fn: F, arch: &ArchConstraints) -> Result<(), + GpuError>' + status: implemented + notes: GGUF loader path; validates via ValidatedLayerWeights::validate() +- contract: architecture-requirements-v1.yaml + equation: apr_loader_validation + module_path: realizar::apr::cuda_part_02_part_03 + function: AprV2ModelCuda::upload_quantized_weights_to_gpu + signature: fn upload_quantized_weights_to_gpu(&mut self) -> Result + status: implemented + notes: APR loader path; derives ArchConstraints from metadata +- contract: architecture-requirements-v1.yaml + equation: safetensors_loader_validation + module_path: realizar::safetensors_cuda + function: SafeTensorsCudaModel::upload_weights + signature: fn upload_weights(...) -> Result<(Vec, ...)> + status: implemented + notes: SafeTensors loader; validates completeness + model_load_basic before GPU init +- contract: ptx-target-parity-v1.yaml + equation: target_parity + module_path: realizar::cuda::gpu_profile + function: GpuProfile::detect + signature: 'fn detect(context: &CudaContext) -> GpuProfile' + status: implemented + notes: sm_target set from context.compute_capability() at executor init +- contract: ptx-target-parity-v1.yaml + equation: no_hardcoded_targets + module_path: realizar::cuda::kernel_generator + function: CudaKernels::generate_ptx + signature: 'fn generate_ptx(&self, kernel_type: &KernelType) -> String' + status: implemented + notes: All PTX generation uses self.sm_target — no emit_ptx() calls in executor/ +- contract: ptx-target-parity-v1.yaml + equation: jit_compilation_success + module_path: realizar::cuda::executor::core + function: CudaExecutor::compile_ptx + signature: 'fn compile_ptx(&self, ptx: &str) -> Result' + status: implemented + notes: PTX JIT with blocklist; target parity ensures CUDA_SUCCESS +- contract: fused-qkv-projection-v1.yaml + equation: shared_q8_qkv + module_path: realizar::cuda::executor::q4k_mwv_gemv + function: CudaExecutor::batched_qkv_dp4a_q4k_gemv_into + signature: 'fn batched_qkv_dp4a_q4k_gemv_into(&mut self, q_w: u64, k_w: u64, v_w: u64, input: &GpuBuffer, q_out: &GpuBuffer, + k_out: &GpuBuffer, v_out: &GpuBuffer, m: u32, q_dim: u32, kv_dim: u32, k: u32) -> Result<(), GpuError>' + status: implemented + notes: 'PMAT-054A: Q8_1 quantize once, 3x GEMV reuse. Saves 56 kernel launches per token.' +- contract: paged-kv-cache-v1.yaml + equation: slot_mapping + module_path: realizar::cuda::executor::kv_cache + function: PagedKvCache::slot + signature: 'fn slot(&self, req: usize, pos: usize) -> usize' + status: implemented + notes: 'PMAT-052: Requires block table allocator. Currently stride-based.' +- contract: paged-kv-cache-v1.yaml + equation: block_allocation + module_path: realizar::cuda::executor::kv_cache + function: BlockAllocator::allocate + signature: 'fn allocate(&mut self, num_blocks: usize) -> Result, GpuError>' + status: implemented + notes: 'PMAT-052: Free list block allocator. Currently contiguous allocation.' +- contract: paged-kv-cache-v1.yaml + equation: paged_contiguous_equivalence + module_path: realizar::cuda::executor::attention + function: paged_attention_into + signature: 'fn paged_attention_into(&self, q: &GpuBuffer, block_table: &GpuBuffer, ...) -> Result<(), GpuError>' + status: implemented + notes: 'PMAT-053: Paged attention kernel. Currently incremental_attention_into.' +- contract: continuous-batching-v1.yaml + equation: token_budget + module_path: realizar::api::cuda_batch_scheduler + function: CudaBatchScheduler::schedule_step + signature: fn schedule_step(&mut self) -> SchedulerOutput + status: implemented + notes: 'PMAT-051: Token budget scheduler. Currently 10ms accumulation window.' +- contract: continuous-batching-v1.yaml + equation: chunked_prefill + module_path: realizar::cuda::executor::layers::prefill + function: CudaExecutor::chunked_prefill + signature: 'fn chunked_prefill(&mut self, tokens: &[u32], chunk_size: usize) -> Result<(), GpuError>' + status: implemented + notes: 'PMAT-051: Chunked prefill to avoid blocking decode. Currently full prefill.' +- contract: continuous-batching-v1.yaml + equation: correctness_under_batching + module_path: realizar::gguf::cuda::generate_batched_streaming + function: generate_batched_streaming + signature: 'fn generate_batched_streaming(&mut self, slots: &[BatchSlot]) -> Result<(), GpuError>' + status: implemented + notes: 'CORRECTNESS-013: FALSIFIED. c=4 slots 1-3 produce constant tokens per step. + + Confirmed in BOTH HGEMM (token 585) and DP4A (tokens 198/88648/198) paths. + + Root cause in common path: prefill_multi_prompt → decode_batched transition. + + PMAT-070: Fix batched decode frozen slots. PMAT-071: Wire FALSIFY-CB-006. + + Instrument: probador llm test c=1 vs c=4, X-Trace-Level: brick, PMAT051_TRACE=1. + + ' +- contract: tensor-inventory-v1.yaml + equation: tensor_count + module_path: realizar + function: tensor_count + status: implemented + notes: Auto-inferred by pv infer +- contract: q4k-q6k-superblock-v1.yaml + equation: total_bytes + module_path: realizar + function: total_bytes + status: implemented + notes: Auto-inferred by pv infer +- contract: int8-symmetric-quant-v1.yaml + equation: quantize + module_path: realizar + function: quantize + status: implemented + notes: Auto-inferred by pv infer +- contract: matmul-kernel-v1.yaml + equation: matmul + module_path: realizar + function: matmul + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: relu + module_path: realizar + function: relu + status: implemented + notes: Auto-inferred by pv infer +- contract: silu-kernel-v1.yaml + equation: sigmoid + module_path: realizar + function: sigmoid + status: implemented + notes: Auto-inferred by pv infer +- contract: backend-dispatch-v1.yaml + equation: gpu_threshold + module_path: realizar + function: gpu_threshold + status: implemented + notes: Auto-inferred by pv infer +- contract: transpose-kernel-v1.yaml + equation: transpose + module_path: realizar + function: transpose + status: implemented + notes: Auto-inferred by pv infer +- contract: softmax-kernel-v1.yaml + equation: softmax + module_path: realizar + function: softmax + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: gelu + module_path: realizar + function: gelu + status: implemented + notes: Auto-inferred by pv infer +- contract: sliding-window-attention-v1.yaml + equation: effective_context + module_path: realizar + function: effective_context + status: implemented + notes: Auto-inferred by pv infer +- contract: kmeans-kernel-v1.yaml + equation: update + module_path: realizar + function: update + status: implemented + notes: Auto-inferred by pv infer +- contract: gqa-kernel-v1.yaml + equation: gqa + module_path: realizar + function: gqa + status: implemented + notes: Auto-inferred by pv infer +- contract: model-config-algebra-v1.yaml + equation: bounds + module_path: realizar + function: bounds + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: silu + module_path: realizar + function: silu + status: implemented + notes: Auto-inferred by pv infer +- contract: learned-position-embedding-v1.yaml + equation: position_embedding + module_path: realizar + function: position_embedding + status: implemented + notes: Auto-inferred by pv infer +- contract: online-softmax-v1.yaml + equation: standard_softmax + module_path: realizar + function: standard_softmax + status: implemented + notes: Auto-inferred by pv infer +- contract: random-forest-v1.yaml + equation: predict + module_path: realizar + function: predict + status: implemented + notes: Auto-inferred by pv infer +- contract: flash-attention-v1.yaml + equation: flash_attention + module_path: realizar + function: flash_attention + status: implemented + notes: Auto-inferred by pv infer +- contract: rmsnorm-kernel-v1.yaml + equation: rmsnorm + module_path: realizar + function: rmsnorm + status: implemented + notes: Auto-inferred by pv infer +- contract: format-parity-v1.yaml + equation: element_count + module_path: realizar + function: element_count + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: greedy + module_path: realizar + function: greedy + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: top_k + module_path: realizar + function: top_k + status: implemented + notes: Auto-inferred by pv infer +- contract: sampling-algorithms-v1.yaml + equation: top_p + module_path: realizar + function: top_p + status: implemented + notes: Auto-inferred by pv infer +- contract: cma-es-kernel-v1.yaml + equation: sample + module_path: realizar + function: sample + status: implemented + notes: Auto-inferred by pv infer +- contract: rope-extrapolation-v1.yaml + equation: base_frequency + module_path: realizar::layers::position + function: RoPE::new + signature: 'pub fn new(dim: usize, base: f32) -> Result' + status: implemented + notes: 'PMAT-414: freq_i = theta^(-2i/d). Computes inv_freq table at init' +- contract: rope-extrapolation-v1.yaml + equation: rotation_matrix + module_path: realizar::quantize::rope + function: apply_rope_rotation_simd + signature: 'pub fn apply_rope_rotation_simd(x1: &mut [f32], x2: &mut [f32], cos_vals: &[f32], sin_vals: &[f32])' + status: implemented + notes: 'PMAT-414: 2x2 rotation [cos,-sin; sin,cos] on paired dimensions. SIMD path' +- contract: architecture-requirements-v1.yaml + equation: weight_completeness + module_path: realizar::contract_gate + function: validate_completeness + signature: 'fn validate_completeness(arch: &ArchConstraints, present: &[WeightRole], arch_name: &str) -> Result<(), ModelLoadError>' + status: implemented + notes: 'PMAT-414: Checks all required roles present for arch. Returns missing list' +- contract: architecture-requirements-v1.yaml + equation: role_mapping + module_path: realizar::arch_requirements_fallback + function: WeightRole::field_name + signature: 'pub const fn field_name(&self) -> &''static str' + status: implemented + notes: 'PMAT-414: Injective mapping WeightRole → IndexedLayerWeights field name' +- contract: architecture-requirements-v1.yaml + equation: constraint_matrix_exhaustiveness + module_path: realizar::arch_requirements_fallback + function: required_roles + signature: 'pub fn required_roles(arch: &ArchConstraints) -> &''static [WeightRole]' + status: implemented + notes: 'PMAT-414: 4 exhaustive arms for (has_qk_norm, has_bias) boolean pairs' +- contract: qwen35-hybrid-forward-v1.yaml + equation: attention_sublayer + module_path: realizar::gpu::scheduler::forward_from_model + function: forward_block_traced + signature: 'fn forward_block_traced(&mut self, input: &[f32], seq_len: usize, block_idx: usize) -> Result<(Vec, LayerActivation)>' + status: implemented + notes: 'PMAT-414: Full attention sublayer: pre-norm, QKV, RoPE, attention, O proj, residual' +- contract: qwen35-hybrid-forward-v1.yaml + equation: ffn_sublayer + module_path: realizar::quantize::quantize_rmsnorm_into + function: fused_swiglu_simd + signature: 'pub fn fused_swiglu_simd(gate: &mut [f32], up: &[f32])' + status: implemented + notes: 'PMAT-414: SwiGLU FFN kernel. gate *= sigmoid(gate) * up' +- contract: qwen35-hybrid-forward-v1.yaml + equation: hybrid_block + module_path: realizar::safetensors::safetensors_config + function: SafeTensorsConfig::is_hybrid_attention + signature: 'pub fn is_hybrid_attention(&self) -> bool' + status: implemented + notes: 'PMAT-414: Dispatches attention vs GDN per layer_types array' +- contract: qwen35-shapes-v1.yaml + equation: rope_frequency + module_path: realizar::gpu::simd_ops + function: rope_frequency_table + signature: 'fn rope_frequency_table(half_head: usize, head_dim: usize, theta: f32) -> Vec' + status: implemented + notes: 'PMAT-414: freq_i = 1/theta^(2i/head_dim). Used by GPU scheduler' +- contract: quantization-ordering-v1.yaml + equation: size_ordering + module_path: realizar::quantize::format_trait + function: QuantBlockFormat + signature: 'trait QuantBlockFormat { const SUPERBLOCK_BYTES: usize; const ELEMENTS_PER_SUPERBLOCK: usize; }' + status: implemented + notes: 'PMAT-414: Q4K 0.5625 < Q6K 0.8203 < Q8_0 1.0625 B/param' +- contract: fused-qkv-projection-v1.yaml + equation: fused_qkv + module_path: realizar::gpu::scheduler::model_forward_block_gpu + function: fused_qkv_projection + signature: 'pub fn fused_qkv_projection(&mut self, input: &[f32]) -> Result<(Vec, Vec, Vec)>' + status: implemented + notes: 'PMAT-408: Fused QKV projection. CUDA path: fused_qkv_into with DP4A shared Q8' +- contract: sampling-algorithms-v1.yaml + equation: temperature + module_path: realizar::gpu::scheduler::ops + function: sample_topk + signature: 'pub(super) fn sample_topk(logits: &[f32], temperature: f32, top_k: usize) -> usize' + status: implemented + notes: 'PMAT-408: Temperature scaling applied before top-k/top-p sampling' +- contract: bpe-tokenization-v1.yaml + equation: merge_rule + module_path: realizar::gguf::token + function: GgufTokenizer::merge_rules + signature: 'pub fn merge_rules(&self) -> Option>' + status: implemented + notes: 'PMAT-408: BPE merge pairs from GGUF metadata' +- contract: bpe-tokenization-v1.yaml + equation: encode + module_path: realizar::tokenizer + function: Tokenizer::encode + signature: 'pub fn encode(&self, text: &str) -> Vec' + status: implemented + notes: 'PMAT-408: BPE tokenization. Also GGUF path at gguf::token' +- contract: bpe-tokenization-v1.yaml + equation: decode + module_path: realizar::tokenizer + function: Tokenizer::decode + signature: 'pub fn decode(&self, token_ids: &[u32]) -> Result' + status: implemented + notes: 'PMAT-408: BPE detokenization. GPT-2 byte-level BPE for WGPU path' +- contract: continuous-batching-v1.yaml + equation: request_state + module_path: realizar::scheduler::mod_max_scheduler + function: SlotState + signature: 'pub enum SlotState { Idle, Processing, Generating }' + status: implemented + notes: 'PMAT-405: Slot state machine. Also CommandSlotState for GPU path' +- contract: continuous-batching-v1.yaml + equation: scheduling_fairness + module_path: realizar::scheduler::mod_max_scheduler + function: handle_preemption + signature: 'fn handle_preemption(&mut self, output: &mut SchedulerOutput, kv_cache: &mut PagedKvCache)' + status: implemented + notes: 'PMAT-405: Preemption logic for fairness under load' +- contract: inference-pipeline-v1.yaml + equation: residual_stream + module_path: realizar::apply + function: forward_layer + signature: 'fn forward_layer(&mut self, layer_idx: usize, hidden: &[f32]) -> Result>' + status: implemented + notes: 'PMAT-405: hidden += attn_proj, residual += ffn_out. Also GPU PTX residualadd kernel' +- contract: inference-pipeline-v1.yaml + equation: layer_composition + module_path: realizar::gpu::adapters::using + function: forward_layer + signature: 'fn forward_layer(&self, executor: &mut CudaExecutor, input: &GpuBuffer, layer_idx: usize, seq_len: usize) -> Result>' + status: implemented + notes: 'PMAT-405: Chained forward through num_layers. CPU and GPU paths' +- contract: inference-pipeline-v1.yaml + equation: hybrid_layer_schedule + module_path: realizar::safetensors::safetensors_config + function: SafeTensorsConfig::is_hybrid_attention + signature: 'pub fn is_hybrid_attention(&self) -> bool' + status: implemented + notes: 'PMAT-405: GDN/attention dispatch per layer_types array (Qwen3.5 MoE)' +- contract: inference-pipeline-v1.yaml + equation: kv_cache_growth + module_path: realizar::verification_specs + function: verify_kv_cache_size + signature: 'fn verify_kv_cache_size(num_layers: u64, num_heads: u64, head_dim: u64, max_seq_len: u64) -> u64' + status: implemented + notes: 'PMAT-405: KV cache = 2 * layers * seq_len * kv_dim * 4 bytes' +- contract: q4k-q6k-superblock-v1.yaml + equation: q4k_superblock + module_path: realizar::quantize::format_trait + function: Q4K + signature: 'impl QuantBlockFormat for Q4K { ELEMENTS_PER_SUPERBLOCK: 256, SUPERBLOCK_BYTES: 144, QUANT_BITS: 4 }' + status: implemented + notes: 'PMAT-405: 144 bytes/256 elements. 12-byte packed scales+mins' +- contract: q4k-q6k-superblock-v1.yaml + equation: q6k_superblock + module_path: realizar::quantize::format_trait + function: Q6K + signature: 'impl QuantBlockFormat for Q6K { ELEMENTS_PER_SUPERBLOCK: 256, SUPERBLOCK_BYTES: 210, QUANT_BITS: 6 }' + status: implemented + notes: 'PMAT-405: 210 bytes/256 elements. Higher precision for LmHead' +- contract: q4k-q6k-superblock-v1.yaml + equation: dequantization + module_path: realizar::quantize::format_trait + function: Q4K::dequant_value + signature: 'fn dequant_value(superblock: &[u8], i: usize) -> f32' + status: implemented + notes: 'PMAT-405: Per-element Q4K→FP32. Also fused_q4k_dot for dot+dequant' +- contract: q4k-q6k-superblock-v1.yaml + equation: bsum + module_path: realizar::quantize::bsum_precompute + function: precompute_q8k_bsums + signature: 'pub fn precompute_q8k_bsums(q8k_quants: &[i8], num_superblocks: usize) -> Result>' + status: implemented + notes: 'PMAT-405: Block sums for DP4A correction term. 8 sub-blocks per superblock' +- contract: roofline-model-v1.yaml + equation: model_bytes + module_path: realizar::lambda + function: model_size_bytes + signature: 'pub fn model_size_bytes(&self) -> usize' + status: implemented + notes: 'PMAT-405: Total weight bytes for roofline BW calculation' +- contract: roofline-model-v1.yaml + equation: compute_ceiling + module_path: realizar::brick::fused + function: CoalescedDp4aBrick::arithmetic_intensity + signature: 'pub fn arithmetic_intensity(&self) -> f64' + status: implemented + notes: 'PMAT-405: FLOP/byte ratio for roofline compute bound' +- contract: format-parity-v1.yaml + equation: transpose_involution + module_path: realizar::contract_gate + function: transpose_f32 + signature: 'pub fn transpose_f32(data: &[f32], rows: usize, cols: usize) -> Vec' + status: implemented + notes: 'PMAT-405: transpose(transpose(x)) == x. Delegates to trueno::blis::transpose' +- contract: format-parity-v1.yaml + equation: name_bijection + module_path: realizar::tensor_names + function: resolve_global + signature: 'pub(crate) fn resolve_global(source: &S, arch: &str, role: GlobalTensorRole) -> Result>' + status: implemented + notes: 'PMAT-405: Weight name mapping across GGUF/SafeTensors/APR formats' +- contract: backend-dispatch-v1.yaml + equation: simd_only_threshold + module_path: realizar::quantize::fused_k + function: fused_q4k_dot_simd + signature: 'pub fn fused_q4k_dot_simd(q4k_data: &[u8], activations: &[f32]) -> Result' + status: implemented + notes: 'PMAT-405: Runtime AVX2 detection + scalar fallback. 5x PCIe rule for CPU vs GPU' +- contract: backend-dispatch-v1.yaml + equation: garbage_oracle + module_path: realizar::inference_trace::mod_get_top_compute + function: is_garbage_output + signature: 'fn is_garbage_output(text: &str) -> bool' + status: implemented + notes: 'PMAT-405: Detects garbled output from quantization errors' +- contract: backend-dispatch-v1.yaml + equation: qk_norm_score_bound + module_path: realizar::cuda::executor::layers::apply + function: apply_qk_norm + signature: 'fn apply_qk_norm(&mut self, q_buf: &GpuBuffer, k_buf: &GpuBuffer, layer_weights: &ValidatedLayerWeights, epsilon: f32) -> Result<(), GpuError>' + status: implemented + notes: 'PMAT-405: QK normalization bounds attention scores. CPU and GPU paths' +- contract: kv-cache-equivalence-v1.yaml + equation: page_shape + module_path: realizar::paged_kv::mod_paged + function: PagedKvCache::new + signature: 'pub fn new(total_pages: usize, block_size: usize, num_heads: usize, head_dim: usize) -> Self' + status: implemented + notes: 'PMAT-405: KvPage shape [block_size, num_heads, head_dim]. tokens_to_pages rounding' +- contract: gpu-context-health-v1.yaml + equation: fp8_architecture_guard + module_path: realizar::cuda::gpu_profile + function: detect_fp8_prefill + signature: 'fn detect_fp8_prefill(cc: u32) -> bool' + status: implemented + notes: 'PMAT-404: cc >= 89 && cc < 100. Blackwell (cc>=121) excluded.' +- contract: gpu-context-health-v1.yaml + equation: context_health + module_path: realizar::cuda::executor::layers::cublas_prefill::attention + function: CudaExecutor::warmup_fp8_cache + signature: 'pub(crate) fn warmup_fp8_cache(&mut self, num_layers: usize, hidden_dim: u32, ...)' + status: implemented + notes: 'PMAT-404: No-op when cc >= 100. Prevents context poisoning on Blackwell.' +- contract: gpu-context-health-v1.yaml + equation: cuda_graph_guard + module_path: realizar::cuda::executor::layers::graphed_capture + function: try_graph_capture + signature: 'fn try_graph_capture(&mut self) -> bool' + status: implemented + notes: 'PMAT-404: Opt-IN via CUDA_GRAPH_ENABLE=1. ThreadLocal capture mode (PMAT-374).' +- contract: gpu-weight-residency-v1.yaml + equation: throughput_target + module_path: realizar::cuda::executor::weights + function: CudaExecutor::load_weights + signature: 'pub fn load_weights(&mut self, name: &str, weights: &[f32]) -> Result' + status: implemented + notes: 'PMAT-404: Pre-uploads all weights at startup. Uses from_host_registered on cc>=120.' +- contract: gpu-decode-profiling-v1.yaml + equation: graph_disable + module_path: realizar::cuda::executor::layers::graphed_capture + function: try_graph_capture + signature: 'fn try_graph_capture(&mut self) -> bool' + status: implemented + notes: 'PMAT-403: CUDA_GRAPH_ENABLE=1 required. Profiling requires graphs OFF (bricks only measured during capture, not replay)' +- contract: gpu-decode-profiling-v1.yaml + equation: report_fidelity + module_path: realizar::cuda::executor::executor_api + function: ExecutorApi::start_brick_id + signature: 'pub(crate) fn start_brick_id(&mut self, brick_id: BrickId) -> Option' + status: implemented + notes: 'PMAT-403: JSON actual_us must equal profiler per-call avg. start/stop_brick_id provide ground truth' +- contract: gpu-decode-profiling-v1.yaml + equation: report_completeness + module_path: realizar::cuda::executor::executor_api + function: ExecutorApi::stop_brick_id + signature: 'pub(crate) fn stop_brick_id(&mut self, timer: Option, elements: u64)' + status: implemented + notes: 'PMAT-403: Every profiler brick must appear in output. No silent truncation' +- contract: gpu-decode-profiling-v1.yaml + equation: report_denominator + module_path: realizar::cuda::executor::executor_api + function: ExecutorApi::start_brick_id + signature: 'pub(crate) fn start_brick_id(&mut self, brick_id: BrickId) -> Option' + status: implemented + notes: 'PMAT-403: decoded_tokens = LmHead.count (exactly 1 LmHead per decoded token). Never use profiler.total_tokens' +- contract: gpu-decode-profiling-v1.yaml + equation: report_metadata + module_path: realizar::cuda::executor::executor_api + function: ExecutorApi::start_brick_id + signature: 'pub(crate) fn start_brick_id(&mut self, brick_id: BrickId) -> Option' + status: implemented + notes: 'PMAT-403: metadata fields must be measured or zero. No hardcoded nonzero constants (137, 173.9, etc.)' +critical_path: +- forward +- flash_attention +- fused_q4k_dot +- sample +- kv_cache +- quantize +- dequantize diff --git a/contracts/release-readiness-v1.yaml b/contracts/release-readiness-v1.yaml index bf8bc8d8db..01831fed41 100644 --- a/contracts/release-readiness-v1.yaml +++ b/contracts/release-readiness-v1.yaml @@ -25,7 +25,14 @@ # model's measured length. Kernels = every (kernel, quant) on ANY required host's dispatch path, owed on EVERY # required host (each GPU arch). # -# THE FAMILY. `--shape release-readiness-v1` grades these nine shapes, armed whatever `armed_shapes` says, and +# DERIVED CELLS (#3745 S2, #3777). The verb axis is no longer a list: the cells are derived from the release +# candidate's own `apr surface --json` (--surface): per (model command, model, host) a deterministic strength-2 +# covering array over its mode args ∪ {input shape} ∪ (when the surface's typed `generates` says so) {thinking, +# rung}; a model command that does not generate owes `release:ModelCell`s; a command with no model owes one +# `release:ProbeCell` per host; S2.5 adds one-factor-at-a-time `release:EffectCell`s judged by `release:ModeEffect`. +# Receipt rows key onto their cell by `cell_id` (`pv extract --cells-out` lists them). +# +# THE FAMILY. `--shape release-readiness-v1` grades these shapes, armed whatever `armed_shapes` says, and # reports them in this declaration order: release → host → context → model → coverage → tokenizer → kernel → # refusal → cell, so a tokenizer that disagrees with llama.cpp (#3726) or a kernel that diverges across arches is # named before any model cell it breaks (#3715 addenda). @@ -84,6 +91,12 @@ shapes: - {path: release:dogfoodReceipt, minCount: 1, maxCount: 1, nodeKind: IRI, node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [GO]}, {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # #3745 S2: the cells are DERIVED from the release candidate's own surface — no surface, no cells, no pass + - {path: release:surface, minCount: 1, maxCount: 1, nodeKind: IRI, class: cli:Surface} + - {path: release:generatesUndeclared, maxCount: 0} + # the two shrink-only counts (args no role type claims; stdin forms v1 cannot declare) never grow + - {path: release:ratchetGrew, maxCount: 0} + - {path: release:ratchetBaselineMissing, maxCount: 0} - id: release-readiness-v1.host targetClass: release:Host properties: @@ -140,15 +153,71 @@ shapes: {path: release:withinBound, minCount: 1, maxCount: 1, in: [true]}, {path: release:boundSource, minCount: 1}, {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # S2.4 (#3739): every derived verb has a CRUX entry naming every declared engine — a counterpart or `none` + reason + - id: release-readiness-v1.crux-verb + targetClass: release:CruxVerb + properties: + - {path: release:mappingEntry, minCount: 1, maxCount: 1} + - {path: release:engineMissing, maxCount: 0} + - {path: release:noneWithoutReason, maxCount: 0} + # ... and every (host, model, mapped verb, thinking, rung) was compared: RED, UNJUDGED and absent are violations; + # ALL_WRONG is named, not a violation, pending the cop's ruling + - id: release-readiness-v1.crux + targetClass: release:CruxObligation + properties: + # a verb whose correspondence entry declares request modes owes a :CruxCell in each (#3739 slice 4) + - {path: release:modeMissing, maxCount: 0} + - {path: release:cruxCell, minCount: 1, nodeKind: IRI, + node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [GREEN, ALL_WRONG]}, + {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # S2.5, the flag-effect oracle: every (command, mode arg, level) is OBSERVED to change the output somewhere — + # an effect cell's output digest differs from its base's (#3745 issuecomment-5765674584) + - id: release-readiness-v1.effect + targetClass: release:ModeEffect + properties: + - {path: release:observedIn, minCount: 1, nodeKind: IRI} + # S2.5, a8's sampling controls over typed `sampling` args: sampled ≠ greedy, the same seed ×2 identical, seed A ≠ + # seed B, top-k 1 == T 0; a control the command cannot express (no typed knob) is named, never passed + - id: release-readiness-v1.sampling + targetClass: release:SamplingCheck + properties: + - {path: release:sampledDiffers, maxCount: 1, in: [true]} + - {path: release:seedRepeatable, maxCount: 1, in: [true]} + - {path: release:seedsDiffer, maxCount: 1, in: [true]} + - {path: release:greedyAgrees, maxCount: 1, in: [true]} + - {path: release:underivable, maxCount: 0} # a cell that does not FIT the host is owed as an HONEST PRE-LOAD REFUSAL naming its arithmetic — never an OOM # mid-run, never a silent truncation, never a pass (#3710 rule) - id: release-readiness-v1.refusal targetClass: release:RefusalCell + allowEmpty: "a RefusalCell exists only where a cell does not fit its host; a release where every cell fits has none (#3610)" properties: - {path: release:row, minCount: 1, maxCount: 1, nodeKind: IRI, node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [refused]}, {path: release:arithmeticNamed, minCount: 1, maxCount: 1, in: [true]}, {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # a command with no model arg: it runs, or refuses by name (#3745 design 3, absorbs #3597) + - id: release-readiness-v1.probe + targetClass: release:ProbeCell + properties: + - {path: release:row, minCount: 1, maxCount: 1, nodeKind: IRI, + node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [pass, refused]}, + {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # a model command that does not generate text (inspect-like): Pass, or a typed refusal by name + - id: release-readiness-v1.model-cell + targetClass: release:ModelCell + properties: + - {path: release:row, minCount: 1, maxCount: 1, nodeKind: IRI, + node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [pass, refused]}, + {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}]}} + # S2.5's one-factor-at-a-time cells: each must run, and say what it printed + - id: release-readiness-v1.effect-cell + targetClass: release:EffectCell + properties: + - {path: release:row, minCount: 1, maxCount: 1, nodeKind: IRI, + node: {properties: [{path: release:verdict, minCount: 1, maxCount: 1, in: [pass]}, + {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}, + {path: release:outputSha, minCount: 1, maxCount: 1}]}} - id: release-readiness-v1 targetClass: release:Cell properties: @@ -159,7 +228,9 @@ shapes: {path: release:fresh, minCount: 1, maxCount: 1, in: [true]}, {path: release:contextMet, minCount: 1, maxCount: 1, in: [true]}, {path: release:thinkOk, minCount: 1, maxCount: 1, in: [true]}, - {path: release:answered, minCount: 1, maxCount: 1, in: [true]}]}} + {path: release:answered, minCount: 1, maxCount: 1, in: [true]}, + # #3748: the CPU reference ran fresh, or came from THIS binary's own receipt + {path: release:f2Measured, minCount: 1, maxCount: 1, in: [true]}]}} equations: cell_universe: diff --git a/contracts/rmsnorm-kernel-v1.yaml b/contracts/rmsnorm-kernel-v1.yaml index a709b49f19..0988adc3c1 100644 --- a/contracts/rmsnorm-kernel-v1.yaml +++ b/contracts/rmsnorm-kernel-v1.yaml @@ -221,7 +221,9 @@ falsification_tests: kani_harnesses: - id: KANI-RN-001 obligation: RN-INV-001 - property: Output always finite with ε > 0 + property: 'Output always finite with ε > 0, |x_i| ≤ 1e18 and |γ_i| ≤ 1e30. Past + |x| ~ 1e19 Σx² overflows and the output collapses to 0 (Kani flagged it via its sqrtf model, PMAT-3140); + |γ| > f32::MAX/√n overflows the output, and 1e30 is a conservative bound under it' bound: 16 strategy: exhaustive harness: verify_rmsnorm_finiteness diff --git a/contracts/shapes.ttl b/contracts/shapes.ttl index 667b5d4245..a460ce585b 100644 --- a/contracts/shapes.ttl +++ b/contracts/shapes.ttl @@ -2,6 +2,18 @@ @prefix xsd: . @prefix ont: . + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; +. + a sh:NodeShape ; sh:targetClass ; sh:property [ @@ -269,6 +281,25 @@ sh:nodeKind sh:IRI ; sh:node ; ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:class ; + sh:nodeKind sh:IRI ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; . a sh:NodeShape ; @@ -475,6 +506,89 @@ ] ; . + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; +. + + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:nodeKind sh:IRI ; + sh:node ; + ] ; +. + + a sh:NodeShape ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "GREEN" "ALL_WRONG" ) ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; +. + + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:nodeKind sh:IRI ; + ] ; +. + + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; + sh:property [ + sh:path ; + sh:maxCount 0 ; + ] ; +. + a sh:NodeShape ; sh:targetClass ; sh:property [ @@ -507,6 +621,89 @@ ] ; . + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:nodeKind sh:IRI ; + sh:node ; + ] ; +. + + a sh:NodeShape ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "pass" "refused" ) ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; +. + + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:nodeKind sh:IRI ; + sh:node ; + ] ; +. + + a sh:NodeShape ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "pass" "refused" ) ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; +. + + a sh:NodeShape ; + sh:targetClass ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:nodeKind sh:IRI ; + sh:node ; + ] ; +. + + a sh:NodeShape ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "pass" ) ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + ] ; +. + a sh:NodeShape ; sh:targetClass ; sh:property [ @@ -561,5 +758,11 @@ sh:maxCount 1 ; sh:in ( "true"^^ ) ; ] ; + sh:property [ + sh:path ; + sh:minCount 1 ; + sh:maxCount 1 ; + sh:in ( "true"^^ ) ; + ] ; . diff --git a/contracts/simular/binding.yaml b/contracts/simular/binding.yaml new file mode 100644 index 0000000000..69f0e2cec7 --- /dev/null +++ b/contracts/simular/binding.yaml @@ -0,0 +1,108 @@ +version: 1.0.0 +target_crate: simular +critical_path: +- step +- simulate +- seek +- record_step +bindings: +- contract: iterator-v1.yaml + equation: step + function: step + module_path: simular::src::tui::tsp_app::step + status: implemented +- contract: iterator-v1.yaml + equation: step + function: new + module_path: simular::src::edd::audit::new + status: implemented +- contract: iterator-v1.yaml + equation: step + function: seek + module_path: simular::src::edd::audit::seek + status: implemented +- contract: qwen35-hybrid-forward-v1.yaml + equation: step + function: step_forward + module_path: simular::src::edd::audit::step_forward + status: implemented +- contract: iterator-v1.yaml + equation: step + function: find_by_step_id + module_path: simular::src::edd::audit::find_by_step_id + status: implemented +- contract: iterator-v1.yaml + equation: step + function: record_step + module_path: simular::src::edd::v2::record_step + status: implemented +- contract: iterator-v1.yaml + equation: step + function: record_step_with_equations + module_path: simular::src::edd::v2::record_step_with_equations + status: implemented +- contract: iterator-v1.yaml + equation: step + function: step_count + module_path: simular::src::edd::v2::step_count + status: implemented +- contract: iterator-v1.yaml + equation: step + function: register_journey + module_path: simular::src::edd::gui_coverage::register_journey + status: implemented +- contract: iterator-v1.yaml + equation: step + function: timestep_secs + module_path: simular::src::engine::clock::timestep_secs + status: implemented +- contract: iterator-v1.yaml + equation: step + function: steps_until + module_path: simular::src::engine::clock::steps_until + status: implemented +- contract: iterator-v1.yaml + equation: step + function: substep_multiplier + module_path: simular::src::orbit::heijunka::substep_multiplier + status: implemented +- contract: iterator-v1.yaml + equation: step + function: estimate_substeps + module_path: simular::src::orbit::heijunka::estimate_substeps + status: implemented +- contract: iterator-v1.yaml + equation: step + function: step_days + module_path: simular::src::orbit::wasm::step_days + status: implemented +- contract: iterator-v1.yaml + equation: step + function: step_hours + module_path: simular::src::orbit::wasm::step_hours + status: implemented +- contract: iterator-v1.yaml + equation: step + function: run_steps + module_path: simular::src::orbit::wasm::run_steps + status: implemented +- contract: iterator-v1.yaml + equation: step + function: run_periods + module_path: simular::src::demos::harmonic_oscillator::run_periods + status: implemented +- contract: iterator-v1.yaml + equation: step + function: run_orbits + module_path: simular::src::demos::kepler_orbit::run_orbits + status: implemented +- contract: iterator-v1.yaml + equation: simulate + function: simulate_path + module_path: simular::src::scenarios::portfolio::simulate_path + status: implemented +- contract: iterator-v1.yaml + equation: simulate + function: simulate + module_path: simular::src::domains::ml::mod::simulate + status: implemented diff --git a/contracts/trueno-db/binding.yaml b/contracts/trueno-db/binding.yaml new file mode 100644 index 0000000000..280defc528 --- /dev/null +++ b/contracts/trueno-db/binding.yaml @@ -0,0 +1,16 @@ +version: 1.0.0 +target_crate: trueno-db +critical_path: +- query +- get_or_insert +bindings: +- contract: configuration-v1.yaml + equation: query + function: query + module_path: trueno_db::src::wasm::query + status: implemented +- contract: configuration-v1.yaml + equation: insert + function: get_or_insert + module_path: trueno_db::src::gpu::jit::get_or_insert + status: implemented diff --git a/contracts/trueno-rag/binding.yaml b/contracts/trueno-rag/binding.yaml new file mode 100644 index 0000000000..a419e3427c --- /dev/null +++ b/contracts/trueno-rag/binding.yaml @@ -0,0 +1,108 @@ +version: 1.0.0 +target_crate: trueno-rag +critical_path: +- embed +- retrieve +- chunk +- insert +bindings: +- contract: embedding-algebra-v1.yaml + equation: embed + function: insert + module_path: trueno_rag::src::multivector::index::insert + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: find_nearest_centroid + module_path: trueno_rag::src::multivector::codec::find_nearest_centroid + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: compress + module_path: trueno_rag::src::multivector::codec::compress + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: train + module_path: trueno_rag::src::multivector::codec::train + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: new + module_path: trueno_rag::src::multivector::types::new + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: embedder + module_path: trueno_rag::src::retrieve::embedder + status: implemented +- contract: embedding-algebra-v1.yaml + equation: embed + function: set_embedding + module_path: trueno_rag::src::chunk::mod::set_embedding + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: retrieve + module_path: trueno_rag::src::retrieve::retrieve + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: retrieve_dense + module_path: trueno_rag::src::retrieve::retrieve_dense + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: retrieve_sparse + module_path: trueno_rag::src::retrieve::retrieve_sparse + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: compute + module_path: trueno_rag::src::metrics::compute + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: recall_at_k + module_path: trueno_rag::src::metrics::recall_at_k + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: precision_at_k + module_path: trueno_rag::src::metrics::precision_at_k + status: implemented +- contract: pagerank-kernel-v1.yaml + equation: retrieve + function: mean_reciprocal_rank + module_path: trueno_rag::src::metrics::mean_reciprocal_rank + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: ndcg_at_k + module_path: trueno_rag::src::metrics::ndcg_at_k + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: average_precision + module_path: trueno_rag::src::metrics::average_precision + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: f1_at_k + module_path: trueno_rag::src::metrics::f1_at_k + status: implemented +- contract: configuration-v1.yaml + equation: retrieve + function: hit_rate_at_k + module_path: trueno_rag::src::metrics::hit_rate_at_k + status: implemented +- contract: configuration-v1.yaml + equation: chunk + function: chunk_count + module_path: trueno_rag::src::sqlite::mod::chunk_count + status: implemented +- contract: configuration-v1.yaml + equation: chunk + function: get_chunk + module_path: trueno_rag::src::sqlite::mod::get_chunk + status: implemented diff --git a/contracts/trueno/binding.yaml b/contracts/trueno/binding.yaml new file mode 100644 index 0000000000..32a810f4ba --- /dev/null +++ b/contracts/trueno/binding.yaml @@ -0,0 +1,336 @@ +version: 1.0.0 +target_crate: trueno +bindings: +- contract: wgpu-forward-pass-v1.yaml + equation: rmsnorm_correctness + module_path: trueno::backends::gpu::device::linalg::wgsl_forward + function: forward_model + status: partial + notes: '#4369: #[contract] site at crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs:753 had no binding row, and contracts/wgpu-forward-pass-v1.yaml does not exist in-tree: the annotation cites no contract' +- contract: pipeline-cache-v1.yaml + equation: pipeline_cache + module_path: trueno::backends::gpu::batch::execute::dispatch + function: PipelineCache + signature: 'pub type PipelineCache = HashMap' + status: implemented + notes: 'Type alias + cache_key() pattern; get-or-insert via contains_key + conditional + insert in execute_inner()' +- contract: pipeline-cache-v1.yaml + equation: single_encoder + module_path: trueno::backends::gpu::batch::execute + function: GpuCommandBatch::execute + signature: 'pub async fn execute(&mut self) -> Result<(), String>' + status: implemented + notes: 'Single command encoder per execute() call; N ops batched (C-BATCH-EXEC-001)' +- contract: tiled-matmul-shader-v1.yaml + equation: matmul_shader + module_path: trueno::backends::gpu::shaders::basic_ops + function: MATMUL_SHADER + signature: "pub const MATMUL_SHADER: &str" + status: implemented + notes: '16x16 tiled WGSL shader with shared memory (KAIZEN-021, C-TILED-MATMUL-001)' +- contract: tiled-matmul-shader-v1.yaml + equation: matmul_dispatch + module_path: trueno::backends::gpu::device::linalg::matmul + function: matmul_async + signature: 'fn matmul_async(a: &GpuBuffer, b: &GpuBuffer, m: u32, n: u32, k: u32) + -> GpuBuffer' + status: implemented + notes: 'Dispatch: ceil(M/16) x ceil(N/16) x 1 workgroups' +- contract: tiled-matmul-shader-v1.yaml + equation: execute_matmul_op + module_path: trueno::backends::gpu::batch::execute::dispatch + function: encode_matmul_op + signature: 'pub(crate) fn encode_matmul_op(&self, encoder: &mut CommandEncoder, cache: + &mut PipelineCache, shader_source: &str, label: &str, a: &BufferId, b: &BufferId, + output: &BufferId, m: u32, k: u32, n: u32) -> Result<(), String>' + status: implemented + notes: 'Named encode_matmul_op in source (encoder pattern, not direct execute)' +- contract: gemm-backward-tiled-v1.yaml + equation: backward_a_gemm + module_path: trueno_gpu::kernels::backward::gemm + function: GemmBackwardAKernel::tiled + signature: 'pub const fn tiled(m: u32, n: u32, k: u32, tile_size: u32) -> Self' + status: implemented + notes: 'grad_A = grad_C @ B^T; tiled with shared memory (C-TILE-BWD-001). Outputs + gemm_backward_a_tiled kernel.' +- contract: gemm-backward-tiled-v1.yaml + equation: backward_b_gemm + module_path: trueno_gpu::kernels::backward::gemm + function: GemmBackwardBKernel::tiled + signature: 'pub const fn tiled(m: u32, n: u32, k: u32, tile_size: u32) -> Self' + status: implemented + notes: 'grad_B = A^T @ grad_C; tiled with shared memory. Outputs gemm_backward_b_tiled + kernel.' +- contract: gemm-backward-tiled-v1.yaml + equation: tiled_gemm_arithmetic_intensity + module_path: trueno_gpu::kernels::backward::gemm + function: GemmBackwardAKernel::tiled + signature: 'pub const fn tiled(m: u32, n: u32, k: u32, tile_size: u32) -> Self' + status: implemented + notes: 'Arithmetic intensity is a computed property of the tiled kernel variant (naive + ~0.5 vs tiled TILE/4 FLOP/byte). No separate accessor; the tiled constructor + implicitly sets this via tile_size parameter.' +- contract: gemm-backward-tiled-v1.yaml + equation: shared_memory_per_tile + module_path: trueno_gpu::kernels::backward::gemm + function: GemmBackwardAKernel::tiled + signature: 'pub const fn tiled(m: u32, n: u32, k: u32, tile_size: u32) -> Self' + status: implemented + notes: 'Shared memory = 2 * tile_size^2 * sizeof(f32) bytes per tile. Computed from + tile_size parameter, not a separate accessor.' +- contract: gemm-backward-tiled-v1.yaml + equation: unrolled_instruction_ratio + module_path: trueno_gpu::kernels::backward::gemm + function: GemmBackwardAKernel::tiled_unrolled + signature: 'pub const fn tiled_unrolled(m: u32, n: u32, k: u32, tile_size: u32) -> + Self' + status: implemented + notes: '4x unrolled variant; IPC ratio 0.57. Both GemmBackwardAKernel and GemmBackwardBKernel + have tiled_unrolled constructors.' +- contract: activation-kernel-v1.yaml + equation: relu + module_path: trueno + function: relu + status: implemented + notes: Auto-inferred by pv infer +- contract: silu-kernel-v1.yaml + equation: sigmoid + module_path: trueno + function: sigmoid + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: gelu + module_path: trueno + function: gelu + status: implemented + notes: Auto-inferred by pv infer +- contract: softmax-kernel-v1.yaml + equation: softmax + module_path: trueno + function: softmax + status: implemented + notes: Auto-inferred by pv infer +- contract: cross-entropy-kernel-v1.yaml + equation: log_softmax + module_path: trueno + function: log_softmax + status: implemented + notes: Auto-inferred by pv infer +- contract: matmul-kernel-v1.yaml + equation: matmul + module_path: trueno + function: matmul + status: implemented + notes: Auto-inferred by pv infer +- contract: gated-delta-net-v1.yaml + equation: read + module_path: trueno + function: read + status: implemented + notes: Auto-inferred by pv infer +- contract: gated-delta-net-v1.yaml + equation: write + module_path: trueno + function: write + status: implemented + notes: Auto-inferred by pv infer +- contract: model-config-algebra-v1.yaml + equation: bounds + module_path: trueno + function: bounds + status: implemented + notes: Auto-inferred by pv infer +- contract: activation-kernel-v1.yaml + equation: silu + module_path: trueno + function: silu + status: implemented + notes: Auto-inferred by pv infer +- contract: random-forest-v1.yaml + equation: predict + module_path: trueno + function: predict + status: implemented + notes: Auto-inferred by pv infer +- contract: kmeans-kernel-v1.yaml + equation: update + module_path: trueno + function: update + status: implemented + notes: Auto-inferred by pv infer +- contract: quantization-ordering-v1.yaml + equation: bytes_per_param + module_path: trueno + function: bytes_per_param + status: implemented + notes: Auto-inferred by pv infer +- contract: embedding-lookup-v1.yaml + equation: embedding_lookup + module_path: trueno + function: embedding_lookup + status: implemented + notes: Auto-inferred by pv infer +- contract: graph-centrality-v1.yaml + equation: eigenvector + module_path: trueno + function: eigenvector + status: implemented + notes: Auto-inferred by pv infer +- contract: tensor-inventory-v1.yaml + equation: tensor_count + module_path: trueno::cbtop::quantize::gguf + function: GgufReader::tensor_count + signature: 'pub fn tensor_count(&self) -> u64' + status: implemented + notes: 'In crates/cbtop/src/quantize/gguf.rs. Counts tensors in GGUF model files.' +- contract: metrics-classification-v1.yaml + equation: precision + module_path: trueno::cbtop::adaptive_ml::types + function: ClassificationMetrics::precision + signature: 'pub fn precision(&self) -> f64' + status: implemented + notes: 'In crates/cbtop/src/adaptive_ml/types.rs. TP / (TP + FP).' +- contract: metrics-classification-v1.yaml + equation: recall + module_path: trueno::cbtop::adaptive_ml::types + function: ClassificationMetrics::recall + signature: 'pub fn recall(&self) -> f64' + status: implemented + notes: 'In crates/cbtop/src/adaptive_ml/types.rs. TP / (TP + FN).' +- contract: attention-kernel-v1.yaml + equation: attention + module_path: trueno_gpu::kernels::attention + function: AttentionKernel + signature: 'pub struct AttentionKernel (FlashAttention variants)' + status: implemented + notes: 'trueno-gpu/src/kernels/attention/mod.rs. FlashAttention (prefill) + Paged + (incremental) + MultiWarp + FlashDecoding variants.' +- contract: int8-symmetric-quant-v1.yaml + equation: quantize + module_path: trueno::crates::trueno_quant::quantize + function: quantize_q4_k + signature: 'pub fn quantize_q4_k(data: &[f32]) -> Vec' + status: implemented + notes: 'crates/trueno-quant/src/quantize.rs. Q4_K/Q5_K/Q6_K per-block scale-min + quantization. Also NF4 in src/brick/quant_ops/nf4.rs.' +- contract: roofline-model-v1.yaml + equation: bandwidth_ceiling + module_path: trueno::monitor::compute_device + function: ComputeDevice::memory_bandwidth_gbps + signature: 'fn memory_bandwidth_gbps(&self) -> anyhow::Result' + status: implemented + notes: 'Trait method on ComputeDevice (TRUENO-SPEC-020). Also in crates/cbtop/src/roofline.rs + HardwareProfile::peak_bandwidth_gbps.' +- contract: roofline-model-v1.yaml + equation: throughput_bound + module_path: trueno::cbtop::roofline + function: HardwareProfile::theoretical_peak_at_oi + signature: 'pub fn theoretical_peak_at_oi(&self, operational_intensity: f64) -> f64' + status: implemented + notes: 'Roofline model: min(compute_ceiling, bandwidth * operational_intensity). + classify_bottleneck() returns compute-bound vs memory-bound.' +- contract: gpu-context-health-v1.yaml + equation: culink_skip + module_path: trueno_gpu::driver::module + function: compile_ptx_to_cubin + signature: 'fn compile_ptx_to_cubin(driver: &CudaDriver, ptx: &str, jit_target: + c_uint) -> Result, GpuError>' + status: implemented + notes: 'PMAT-404: Always returns Err — cuLinkCreate skipped (PMAT-369). Legacy JIT + used.' +- contract: gpu-weight-residency-v1.yaml + equation: pcie_overhead + module_path: trueno_gpu::driver::memory::buffer + function: GpuBuffer::from_host_registered + signature: 'pub unsafe fn from_host_registered(host_ptr: *mut T, len: usize) -> + Result' + status: implemented + notes: 'PMAT-404: Zero-copy via cuMemHostRegister + cuMemHostGetDevicePointer. Eliminates + PCIe transfer on unified memory (cc>=120). PMAT-396/PMAT-420.' +- contract: gpu-decode-profiling-v1.yaml + equation: wall_coverage + module_path: trueno::brick::profiler + function: BrickProfiler::total_ns + signature: 'pub fn total_ns(&self) -> u64' + status: implemented + notes: 'PMAT-403: Total nanoseconds across all brick profiling. sum(brick_total_ns) + / wall_clock_ns >= 0.85 (PAR-200 fast path).' +- contract: gpu-decode-profiling-v1.yaml + equation: sync_verification + module_path: trueno::brick::profiler + function: BrickProfiler::set_sync_mode + signature: 'pub fn set_sync_mode(&mut self, mode: SyncMode)' + status: implemented + notes: 'PMAT-403: Modes: Immediate, PerLayer, Deferred, None. Deferred is default + for production (PAR-200).' +- contract: gpu-decode-profiling-v1.yaml + equation: token_accounting + module_path: trueno::brick::profiler + function: BrickProfiler::total_tokens + signature: 'pub fn total_tokens(&self) -> u64' + status: implemented + notes: 'PMAT-403: Tracks total elements processed. LmHead.count = decoded tokens.' +- contract: gpu-decode-profiling-v1.yaml + equation: brick_ordering + module_path: trueno::brick::profiler + function: BrickProfiler::brick_stats + signature: 'pub fn brick_stats(&self, brick_id: BrickId) -> &BrickStats' + status: implemented + notes: 'PMAT-403: O(1) per-brick access via BrickId array index (PAR-200). Per-call + avg respects kernel complexity (LmHead > Gate > RmsNorm).' +- contract: avx512-blis-v1.yaml + equation: flops_per_tile + module_path: trueno::blis::compute + function: gemm_blis_avx512_large + signature: 'unsafe fn gemm_blis_avx512_large(m: usize, n: usize, k: usize, a: &[f32], b: &[f32], c: &mut [f32]) -> Result<(), TruenoError>' + status: implemented + notes: 'PMAT-037: AVX-512 8×16 microkernel with zmm registers. MR=8, NR=16. BlisProfiler NOT yet wired (C-AVX512-PROF-001 PENDING).' +- contract: avx512-blis-v1.yaml + equation: peak_throughput + module_path: trueno::blis::compute + function: avx512_microkernel_8x16_rowmajor + signature: 'pub(super) unsafe fn avx512_microkernel_8x16_rowmajor(k: usize, a: *const f32, b: *const f32, c: *mut f32, ldc: usize)' + status: implemented + notes: '8 zmm accumulators, B as zmm load, A as scalar broadcast. 128 GFLOPS measured (98.5% of peak).' +- contract: avx512-blis-v1.yaml + equation: flops_per_tile + module_path: trueno::blis::compute + function: pack_b_block_nr16 + signature: 'pub(super) fn pack_b_block_nr16(b: &[f32], ldb: usize, pc: usize, jc: usize, kc: usize, nc: usize, packed: &mut [f32])' + status: implemented + notes: 'NR=16 B packing for AVX-512 zmm width. Zero-pads when nc < 16.' +- contract: blis-thread-cap-v1.yaml + equation: amdahl_speedup + module_path: trueno::blis::parallel + function: gemm_blis_parallel + signature: 'pub fn gemm_blis_parallel(m: usize, n: usize, k: usize, a: &[f32], b: &[f32], c: &mut [f32]) -> Result<(), TruenoError>' + status: implemented + notes: 'PMAT-037: 4-tier thread cap (2/4/phys÷2/all) based on FLOPs thresholds from cgp scaling measurements.' +- contract: avx512-q4k-v1.yaml + equation: dequant + module_path: trueno::backends::q4k::gemv::avx512 + function: matmul_q4k_f32_avx512 + signature: 'pub(crate) unsafe fn matmul_q4k_f32_avx512(q4k_data: &[u8], input: &[f32], out_dim: usize, in_dim: usize) -> Vec' + status: implemented + notes: '16-wide AVX-512 dequant+FMA. +5-35% vs AVX2. Bottleneck: scalar header parsing per super-block.' +- contract: avx512-q4k-v1.yaml + equation: dequant + module_path: trueno::backends::q4k::gemv::avx512 + function: compute_chunk_q4k_avx512 + signature: 'pub(crate) unsafe fn compute_chunk_q4k_avx512(...)' + status: implemented + notes: 'Parallel dispatch chunk processor for AVX-512 Q4K path.' +critical_path: +- softmax +- matmul +- silu +- gelu +- execute_matmul +- PipelineCache +- GemmBackward +- arithmetic_intensity +- gemm_blis_avx512_large +- gemm_blis_parallel +- matmul_q4k_f32_avx512 diff --git a/crates/apr-cli/Cargo.toml b/crates/apr-cli/Cargo.toml index 9a48234091..3cfbb407d0 100644 --- a/crates/apr-cli/Cargo.toml +++ b/crates/apr-cli/Cargo.toml @@ -106,7 +106,10 @@ cuda = ["inference", "realizar/cuda", "entrenar/cuda", "trueno/cuda", "aprender- # the binary was built with it (§2.1 INVALID-BUILD reads that entry). cuda-batch = ["cuda"] # PMAT-335: WGPU backend for AMD/Intel/Apple GPU inference via Vulkan/Metal/WebGPU -wgpu = ["inference"] +# #4056: the finetune wgpu path (commands/finetune.rs) uses entrenar items gated +# behind entrenar's own `gpu` feature. Weak dep: enabled only when `training` +# brings entrenar in, which is exactly when that path compiles. +wgpu = ["inference", "entrenar?/gpu"] visualization = ["renacer", "trueno-viz"] zram = ["trueno-zram-core"] training = ["dep:entrenar", "dep:entrenar-lora", "dep:aprender-train-distill", "dep:aprender-train-common", "aprender-train-distill?/shard-batch-source"] @@ -136,7 +139,7 @@ async-stream = "0.3" provable-contracts-macros = { workspace = true } # Shared formatting and system utilities (Batuta stack) -batuta-common = { workspace = true } +batuta-common = { workspace = true, features = ["cli-roles"] } # Sovereign coding assistant (apr code → batuta agent engine) — PMAT-182. # Non-optional since claude-code-parity-apr M150 (outcome-parity Phase 3 @@ -174,7 +177,7 @@ gag = "1.0" # Output formatting colored = "2.1" -tabled = "0.16" +tabled = "0.22" serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" sha2 = "0.10" @@ -258,6 +261,8 @@ alimentar = { workspace = true } # `dispatch` fn those binaries call, so the surfaces cannot drift. aprender-rag-cli = { workspace = true } aprender-zram-cli = { workspace = true } +# `apr ptx-debug` (#4062): pure-Rust PTX falsification analyzer, always built. +aprender-ptx-debug = { workspace = true } # simular: hand-rolled parser, wired as a verbatim passthrough (apr sim ...) simular = { workspace = true } aprender-cgp = { workspace = true } @@ -289,6 +294,9 @@ required-features = ["inference", "cuda"] name = "ptx_parity_validation" required-features = ["inference"] +[build-dependencies] +aprender-build-sha = { workspace = true } # #4219: APR_GIT_SHA for --version + [dev-dependencies] jsonschema = "0.28" # apr devices --json validated against contracts/schemas/apr-devices-v1.schema.json (PP-066 R-0a) assert_cmd = "2.0" diff --git a/crates/apr-cli/build.rs b/crates/apr-cli/build.rs index 4667107f37..8ee5028d1e 100644 --- a/crates/apr-cli/build.rs +++ b/crates/apr-cli/build.rs @@ -1,77 +1,13 @@ // Contract: apr-version-traceability-v1 F-VERSION-001..004 (paiml/aprender#597, #1862). // Resolve APR_GIT_SHA with a multi-source fallback hierarchy: // 1. APR_GIT_SHA_OVERRIDE env var (CI/release hook) -// 2. git rev-parse --short HEAD (dev builds from worktree or primary checkout) -// 3. committed .git-sha file (crates.io installs) -// 4. "v{CARGO_PKG_VERSION}+no-git" (informative fallback — never bare "unknown") +// 2. .cargo_vcs_info.json `git.sha1` (crates.io / packaged builds, #4110) +// 3. git rev-parse --short HEAD, retried with the checkout marked safe.directory (#4110) +// 4. committed .git-sha file +// 5. "v{CARGO_PKG_VERSION}+no-git" (informative fallback — never bare "unknown") +// +// The resolution and its rerun-if-changed triggers live in the shared +// `aprender-build-sha` crate (#4219) so every workspace [[bin]] stamps the same SHA. fn main() { - let sha = resolve_git_sha(); - println!("cargo:rustc-env=APR_GIT_SHA={sha}"); - - // Rerun build when HEAD moves. In a primary checkout `.git` is a directory; - // in a worktree `.git` is a file pointer to `/worktrees//`. - // Resolve the actual git directory(ies) via `git rev-parse` so both layouts - // are watched correctly (#1862). - register_git_rerun_triggers(); - - println!("cargo:rerun-if-env-changed=APR_GIT_SHA_OVERRIDE"); - println!("cargo:rerun-if-changed=.git-sha"); -} - -fn register_git_rerun_triggers() { - // Worktree-local HEAD: /HEAD. In a worktree this is the per-worktree - // HEAD pointer; in a primary checkout it's the same as /HEAD. - if let Some(git_dir) = run_git(&["rev-parse", "--git-dir"]) { - let head = format!("{git_dir}/HEAD"); - if std::path::Path::new(&head).exists() { - println!("cargo:rerun-if-changed={head}"); - } - } - // Shared refs live under /refs/heads/ regardless of layout. - if let Some(common_dir) = run_git(&["rev-parse", "--git-common-dir"]) { - let refs = format!("{common_dir}/refs/heads"); - if std::path::Path::new(&refs).exists() { - println!("cargo:rerun-if-changed={refs}"); - } - } -} - -fn run_git(args: &[&str]) -> Option { - let out = std::process::Command::new("git").args(args).output().ok()?; - if !out.status.success() { - return None; - } - let s = String::from_utf8_lossy(&out.stdout).trim().to_string(); - if s.is_empty() { - None - } else { - Some(s) - } -} - -fn resolve_git_sha() -> String { - // 1. Explicit override from CI/release - if let Ok(s) = std::env::var("APR_GIT_SHA_OVERRIDE") { - let s = s.trim(); - if !s.is_empty() { - return s.to_string(); - } - } - - // 2. Live git hash from worktree (works for both primary checkouts and worktrees) - if let Some(sha) = run_git(&["rev-parse", "--short", "HEAD"]) { - return sha; - } - - // 3. Committed .git-sha fallback (updated by release script at publish time) - if let Ok(contents) = std::fs::read_to_string(".git-sha") { - let s = contents.trim().to_string(); - if !s.is_empty() { - return s; - } - } - - // 4. Informative fallback — never bare "unknown" - let version = std::env::var("CARGO_PKG_VERSION").unwrap_or_else(|_| "0.0.0".to_string()); - format!("v{version}+no-git") + build_sha::emit(); } diff --git a/crates/apr-cli/src/bin/apr-corpus-ingest.rs b/crates/apr-cli/src/bin/apr-corpus-ingest.rs index 221c8f402a..3c53cd9f2b 100644 --- a/crates/apr-cli/src/bin/apr-corpus-ingest.rs +++ b/crates/apr-cli/src/bin/apr-corpus-ingest.rs @@ -52,7 +52,7 @@ const REQUIRED_TOP_KEYS: &[&str] = &[ #[derive(Parser)] #[command( name = "apr-corpus-ingest", - version, + version = concat!(env!("CARGO_PKG_VERSION"), " (", env!("APR_GIT_SHA"), ")"), about = "Dry-run scaffold for SHIP-TWO-001 MODEL-2 corpus ingest (C-DATA-THESTACK-PYTHON)" )] struct Cli { diff --git a/crates/apr-cli/src/commands/canary.rs b/crates/apr-cli/src/commands/canary.rs index fc34168804..473a92b9d1 100644 --- a/crates/apr-cli/src/commands/canary.rs +++ b/crates/apr-cli/src/commands/canary.rs @@ -16,6 +16,7 @@ //! ``` use crate::error::{CliError, Result}; +use batuta_common::cli_roles::{ConfigPath, InputFile, ModelPath, OutputPath}; use clap::Subcommand; use colored::Colorize; use serde::{Deserialize, Serialize}; @@ -30,25 +31,25 @@ pub enum CanaryCommands { Create { /// Model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Input file (e.g. wav) #[arg(long)] - input: PathBuf, + input: InputFile, /// Output json file #[arg(long)] - output: PathBuf, + output: OutputPath, }, /// Check against a canary test Check { /// Model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Canary json file #[arg(long)] - canary: PathBuf, + canary: ConfigPath, }, } diff --git a/crates/apr-cli/src/commands/canary_tests.rs b/crates/apr-cli/src/commands/canary_tests.rs index 96d11e8d3c..8ce9a737a5 100644 --- a/crates/apr-cli/src/commands/canary_tests.rs +++ b/crates/apr-cli/src/commands/canary_tests.rs @@ -255,9 +255,9 @@ fn test_build_failure_message_std_drift() { #[test] fn test_canary_commands_create() { let cmd = CanaryCommands::Create { - file: PathBuf::from("model.safetensors"), - input: PathBuf::from("input.wav"), - output: PathBuf::from("canary.json"), + file: PathBuf::from("model.safetensors").into(), + input: PathBuf::from("input.wav").into(), + output: PathBuf::from("canary.json").into(), }; match cmd { CanaryCommands::Create { @@ -276,8 +276,8 @@ fn test_canary_commands_create() { #[test] fn test_canary_commands_check() { let cmd = CanaryCommands::Check { - file: PathBuf::from("model.safetensors"), - canary: PathBuf::from("canary.json"), + file: PathBuf::from("model.safetensors").into(), + canary: PathBuf::from("canary.json").into(), }; match cmd { CanaryCommands::Check { file, canary } => { @@ -291,9 +291,9 @@ fn test_canary_commands_check() { #[test] fn test_canary_commands_clone() { let cmd = CanaryCommands::Create { - file: PathBuf::from("model.safetensors"), - input: PathBuf::from("input.wav"), - output: PathBuf::from("canary.json"), + file: PathBuf::from("model.safetensors").into(), + input: PathBuf::from("input.wav").into(), + output: PathBuf::from("canary.json").into(), }; let cloned = cmd.clone(); match cloned { @@ -307,8 +307,8 @@ fn test_canary_commands_clone() { #[test] fn test_canary_commands_debug() { let cmd = CanaryCommands::Check { - file: PathBuf::from("model.safetensors"), - canary: PathBuf::from("canary.json"), + file: PathBuf::from("model.safetensors").into(), + canary: PathBuf::from("canary.json").into(), }; let debug = format!("{cmd:?}"); assert!(debug.contains("Check")); @@ -323,9 +323,9 @@ fn test_run_create_model_not_found() { let output = NamedTempFile::with_suffix(".json").expect("create output"); let input = NamedTempFile::with_suffix(".wav").expect("create input"); let cmd = CanaryCommands::Create { - file: PathBuf::from("/nonexistent/model.safetensors"), - input: input.path().to_path_buf(), - output: output.path().to_path_buf(), + file: PathBuf::from("/nonexistent/model.safetensors").into(), + input: input.path().to_path_buf().into(), + output: output.path().to_path_buf().into(), }; let result = run(cmd); assert!(result.is_err()); @@ -341,9 +341,9 @@ fn test_run_create_invalid_model() { let input = NamedTempFile::with_suffix(".wav").expect("create input"); let cmd = CanaryCommands::Create { - file: model.path().to_path_buf(), - input: input.path().to_path_buf(), - output: output.path().to_path_buf(), + file: model.path().to_path_buf().into(), + input: input.path().to_path_buf().into(), + output: output.path().to_path_buf().into(), }; let result = run(cmd); assert!(result.is_err()); @@ -357,8 +357,8 @@ fn test_run_check_model_not_found() { .expect("write"); let cmd = CanaryCommands::Check { - file: PathBuf::from("/nonexistent/model.safetensors"), - canary: canary.path().to_path_buf(), + file: PathBuf::from("/nonexistent/model.safetensors").into(), + canary: canary.path().to_path_buf().into(), }; let result = run(cmd); assert!(result.is_err()); @@ -370,8 +370,8 @@ fn test_run_check_canary_not_found() { model.write_all(b"fake model").expect("write"); let cmd = CanaryCommands::Check { - file: model.path().to_path_buf(), - canary: PathBuf::from("/nonexistent/canary.json"), + file: model.path().to_path_buf().into(), + canary: PathBuf::from("/nonexistent/canary.json").into(), }; let result = run(cmd); assert!(result.is_err()); @@ -385,8 +385,8 @@ fn test_run_check_invalid_canary() { canary.write_all(b"not valid json").expect("write"); let cmd = CanaryCommands::Check { - file: model.path().to_path_buf(), - canary: canary.path().to_path_buf(), + file: model.path().to_path_buf().into(), + canary: canary.path().to_path_buf().into(), }; let result = run(cmd); assert!(result.is_err()); @@ -1169,9 +1169,9 @@ fn test_run_create_input_not_found() { let output = NamedTempFile::with_suffix(".json").expect("create output"); let cmd = CanaryCommands::Create { - file: model.path().to_path_buf(), - input: PathBuf::from("/nonexistent/input.wav"), - output: output.path().to_path_buf(), + file: model.path().to_path_buf().into(), + input: PathBuf::from("/nonexistent/input.wav").into(), + output: output.path().to_path_buf().into(), }; let result = run(cmd); assert!(result.is_err(), "missing input file should error"); @@ -1185,9 +1185,9 @@ fn test_run_create_empty_input_path() { let output = NamedTempFile::with_suffix(".json").expect("create output"); let cmd = CanaryCommands::Create { - file: model.path().to_path_buf(), - input: PathBuf::from(""), - output: output.path().to_path_buf(), + file: model.path().to_path_buf().into(), + input: PathBuf::from("").into(), + output: output.path().to_path_buf().into(), }; // Will fail on format detection, not input validation let result = run(cmd); diff --git a/crates/apr-cli/src/commands/chat.rs b/crates/apr-cli/src/commands/chat.rs index 05fe77bac2..82e6645911 100644 --- a/crates/apr-cli/src/commands/chat.rs +++ b/crates/apr-cli/src/commands/chat.rs @@ -501,8 +501,8 @@ fn clean_chat_response(raw: &str) -> String { trimmed.to_string() } -/// Detect format from magic bytes (more reliable than extension) -#[cfg(feature = "inference")] +/// Detect format from magic bytes (more reliable than extension). Pure, so not feature-gated: its caller +/// `format_from_leading_bytes` is compiled in every build (#4041). fn detect_format_from_bytes(data: &[u8]) -> ModelFormat { if data.len() < 8 { return ModelFormat::Demo; diff --git a/crates/apr-cli/src/commands/diff.rs b/crates/apr-cli/src/commands/diff.rs index e8d77e1148..94b48a057a 100644 --- a/crates/apr-cli/src/commands/diff.rs +++ b/crates/apr-cli/src/commands/diff.rs @@ -245,7 +245,16 @@ pub(crate) fn run( // and run an element-wise diff per `apr-cli-trace-save-tensor-v1` // `apr_diff_values_compat` invariant. if is_aprt_stage_file(path1) && is_aprt_stage_file(path2) { + #[cfg(feature = "inference")] return run_aprt_stage_diff(path1, path2, limit, json_output); + // #4041: the stage-file reader is realizar's. A minimal build names the missing feature + // instead of falling through to the model walker, which would misread an APRT file. + #[cfg(not(feature = "inference"))] + return Err(CliError::ValidationFailed( + "comparing APRT stage tensor files needs the `inference` feature (this apr was built \ + without it)" + .to_string(), + )); } // Run tensor value comparison run_tensor_value_diff(path1, path2, filter, limit, transpose_aware, json_output) diff --git a/crates/apr-cli/src/commands/diff_05_aprt_stage.rs b/crates/apr-cli/src/commands/diff_05_aprt_stage.rs index 7dac722c9e..6cda8dc4c4 100644 --- a/crates/apr-cli/src/commands/diff_05_aprt_stage.rs +++ b/crates/apr-cli/src/commands/diff_05_aprt_stage.rs @@ -91,6 +91,7 @@ fn compute_aprt_stage_stats( // serde_json::json!() macro uses infallible unwrap internally #[allow(clippy::disallowed_methods)] +#[cfg(feature = "inference")] fn run_aprt_stage_diff( path1: &Path, path2: &Path, @@ -223,7 +224,7 @@ fn run_aprt_stage_diff( Ok(()) } -#[cfg(test)] +#[cfg(all(test, feature = "inference"))] mod aprt_stage_diff_tests { use super::*; use std::io::Write; diff --git a/crates/apr-cli/src/commands/explain.rs b/crates/apr-cli/src/commands/explain.rs index ec9d43c293..a1e39bbb44 100644 --- a/crates/apr-cli/src/commands/explain.rs +++ b/crates/apr-cli/src/commands/explain.rs @@ -217,6 +217,20 @@ fn diagnose_missing_model_type(path: &Path, content: &str, json: bool) { } } +/// `name` beside `path`, if it exists. With `inference`, realizar's finder, which also resolves hash-prefixed and +/// symlinked model paths (CB-510). A minimal build (#4041: `--no-default-features` has no realizar) checks the +/// literal sibling only, so a hash-prefixed HF-cache path finds nothing there and reports not-found. +fn sibling_file(path: &Path, name: &str) -> Option { + #[cfg(feature = "inference")] + { + realizar::safetensors::find_sibling_file(path, name) + } + #[cfg(not(feature = "inference"))] + { + path.parent().map(|d| d.join(name)).filter(|p| p.is_file()) + } +} + /// Resolve family from a model file by finding config.json in the same directory. fn resolve_family_from_model_file( path: &Path, @@ -228,7 +242,7 @@ fn resolve_family_from_model_file( // Companion lookup: find_sibling_file handles hash-prefixed / symlinked // model paths robustly (CB-510 / publish-safety) and only returns Some // when the file exists — subsuming the prior with_file_name + .exists(). - if let Some(config_path) = realizar::safetensors::find_sibling_file(&real_path, "config.json") { + if let Some(config_path) = sibling_file(&real_path, "config.json") { resolve_from_config_json(&config_path) } else { emit_kernel_error( @@ -341,7 +355,7 @@ fn resolve_config_mapping( // hash-prefixed / symlinked model paths); fall back to the literal // sibling dir so extract_config_mapping reports not-found as before // (parent().join avoids with_file_name per publish-safety). - realizar::safetensors::find_sibling_file(p, "config.json").unwrap_or_else(|| { + sibling_file(p, "config.json").unwrap_or_else(|| { p.parent().map_or_else( || std::path::PathBuf::from("config.json"), |d| d.join("config.json"), diff --git a/crates/apr-cli/src/commands/finetune.rs b/crates/apr-cli/src/commands/finetune.rs index 48acddf75a..7433cb2332 100644 --- a/crates/apr-cli/src/commands/finetune.rs +++ b/crates/apr-cli/src/commands/finetune.rs @@ -650,6 +650,8 @@ fn execute_training_wgpu( let mut fwd = trueno::backends::gpu::WgslForwardPass::new( gpu.device, gpu.queue, hidden, heads, kv_heads, head_dim, inter, ); + // #4056: the WGSL RMSNorm takes the model's eps (it hardcoded 1e-6). + fwd.set_rms_norm_eps(model_config.rms_norm_eps as f32); // 3. Streaming dequant + upload weights to GPU let lm_head_f32 = dequant_and_upload_weights(&mut fwd, &q_model)?; diff --git a/crates/apr-cli/src/commands/forward_error.rs b/crates/apr-cli/src/commands/forward_error.rs index 52174418a2..43a54d6986 100644 --- a/crates/apr-cli/src/commands/forward_error.rs +++ b/crates/apr-cli/src/commands/forward_error.rs @@ -513,7 +513,7 @@ fn run_format_parity_gate(path: &Path, config: &QaConfig) -> Result #[cfg(feature = "inference")] { use realizar::format::{detect_format, ModelFormat}; - use realizar::gguf::{GGUFModel, MappedGGUFModel, OwnedQuantizedModel}; + use realizar::gguf::{MappedGGUFModel, OwnedQuantizedModel}; // Peek the primary's magic bytes first (cheap — no full-file read) so that // non-GGUF inputs SKIP cleanly instead of churning through SafeTensors @@ -523,8 +523,8 @@ fn run_format_parity_gate(path: &Path, config: &QaConfig) -> Result // silently skip" invariant was scoped to missing-reference failures, not // to category-mismatched inputs. { - let header = std::fs::read(path) - .map(|b| b.into_iter().take(8).collect::>()) + // #3750: this said "cheap — no full-file read" above a whole-file read; now it is 8 bytes + let header = super::model_header::read_prefix(path, 8) .map_err(|e| { CliError::ValidationFailed(format!("Failed to read primary model: {e}")) })?; @@ -546,9 +546,6 @@ fn run_format_parity_gate(path: &Path, config: &QaConfig) -> Result Err(gate_result) => return Ok(gate_result), }; - let gguf_bytes = std::fs::read(path) - .map_err(|e| CliError::ValidationFailed(format!("Failed to read GGUF: {e}")))?; - // Verify SafeTensors model exists if !safetensors_path.exists() { return Ok(GateResult::failed( @@ -563,17 +560,14 @@ fn run_format_parity_gate(path: &Path, config: &QaConfig) -> Result )); } - // Load GGUF model and get tokenizer - let gguf = GGUFModel::from_bytes(&gguf_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; + // Map the GGUF (#3750: the mapped header carries the tokenizer; no second, whole-file read) + let mapped = MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("GGUF map failed: {e}")))?; // Test prompt - use simple arithmetic for deterministic output let prompt = "<|im_start|>user\nWhat is 2+2?<|im_end|>\n<|im_start|>assistant\n"; let bos = aprender::demo::SpecialTokens::qwen2().bos_id; - let prompt_tokens: Vec = gguf.encode(prompt).unwrap_or_else(|| vec![bos, 9707]); - - let mapped = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("GGUF map failed: {e}")))?; + let prompt_tokens: Vec = mapped.model.encode(prompt).unwrap_or_else(|| vec![bos, 9707]); let gguf_model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("GGUF model failed: {e}")))?; diff --git a/crates/apr-cli/src/commands/golden_output.rs b/crates/apr-cli/src/commands/golden_output.rs index c0e338c755..273858dc53 100644 --- a/crates/apr-cli/src/commands/golden_output.rs +++ b/crates/apr-cli/src/commands/golden_output.rs @@ -329,24 +329,26 @@ fn run_golden_output_gate(path: &Path, config: &QaConfig) -> Result #[cfg(feature = "inference")] { use realizar::format::{detect_format, ModelFormat}; - use realizar::gguf::{GGUFModel, MappedGGUFModel}; + use realizar::gguf::MappedGGUFModel; let cuda_available = cuda_device_present(); - let model_bytes = std::fs::read(path) + // #3750: the format from the 8-byte magic and the header from the map, never the whole + // model (it was read into memory here, beside a map of the same file) + let magic = super::model_header::read_prefix(path, 8) .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let format = detect_format(&model_bytes[..8.min(model_bytes.len())]) + let format = detect_format(&magic) .map_err(|e| CliError::ValidationFailed(format!("Failed to detect format: {e}")))?; // GH-239: Only create GGUF objects when format is actually GGUF - let (mapped, gguf_model) = if format == ModelFormat::Gguf { - let m = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; - let g = GGUFModel::from_bytes(&model_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; - (Some(m), Some(g)) + let mapped = if format == ModelFormat::Gguf { + Some( + MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?, + ) } else { - (None, None) + None }; + let gguf_model = mapped.as_ref().map(|m| &m.model); // #3711: the pass message names the GPU leg, so a skipped leg says which skip it was let mut gpu_leg = GpuGoldenLeg::NotRun("no golden case ran"); @@ -358,7 +360,7 @@ fn run_golden_output_gate(path: &Path, config: &QaConfig) -> Result config, format, mapped.as_ref(), - gguf_model.as_ref(), + gguf_model, cuda_available, start, )? { @@ -461,21 +463,18 @@ fn measure_generate_throughput( #[cfg(feature = "inference")] fn throughput_gguf( path: &Path, - model_bytes: &[u8], config: &QaConfig, cuda_available: bool, tracer: &TracerImpl, prompt: &str, ) -> Result<(f64, Duration)> { - use realizar::gguf::{ - GGUFModel, MappedGGUFModel, OwnedQuantizedModel, - QuantizedGenerateConfig, - }; + use realizar::gguf::{MappedGGUFModel, OwnedQuantizedModel, QuantizedGenerateConfig}; - let gguf = GGUFModel::from_bytes(model_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; + // #3750: the tokenizer comes from the mapped header, not a whole-file read + let mapped = MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; let bos = aprender::demo::SpecialTokens::qwen2().bos_id; - let prompt_tokens = gguf.encode(prompt).unwrap_or_else(|| vec![bos, 9707]); + let prompt_tokens = mapped.model.encode(prompt).unwrap_or_else(|| vec![bos, 9707]); let gen_config = QuantizedGenerateConfig { max_tokens: config.max_tokens, temperature: 0.0, @@ -484,8 +483,6 @@ fn throughput_gguf( }; let budget_us = config.max_tokens as u64 * config.iterations as u64 * 100_000; - let mapped = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; let model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("Model failed: {e}")))?; diff --git a/crates/apr-cli/src/commands/gpu_correct_subject.rs b/crates/apr-cli/src/commands/gpu_correct_subject.rs new file mode 100644 index 0000000000..c08a1c7d48 --- /dev/null +++ b/crates/apr-cli/src/commands/gpu_correct_subject.rs @@ -0,0 +1,433 @@ +// #3821: GPU-correct(model_sha256, host, apr_version) is ONE subject, not a +// per-surface opinion. +// +// Two surfaces decide it, in the same binary, on the same host, for the same +// model, and until now neither recorded the other's answer: +// +// * `apr run --gpu` — the F2 parity guard. REFUSES a diverging GPU +// (`BackendUnavailable`, exit 14). +// * `apr qa` `golden_output` — generates on the GPU and scores the text. +// It calls `generate_gpu_resident` directly, so it never reaches the +// guard (#3804: grepping a failing run's stderr for the guard's own lines +// returns zero occurrences). +// +// On qwen2.5-coder-0.5b-instruct-q4_k_m at `a9502d992` the guard refused at +// cosine 0.4153 while the gate shipped the symptom as ``` ``` ``` repetition. +// Both happened to say "bad", so nothing noticed they were never compared. +// The dangerous ordering is the other one: the guard refusing while the gate +// PASSES, which `APR_F2_PROBE_PATH=serial` produces on demand — serial parity +// is cosine >= 0.9978 at all 16 positions, so the guard accepts, and +// generation still runs the batched prefill that emits the fragment. +// +// REPORT_ONLY for 0.69.1 (#3821's ruling): this reconciles and NAMES, it does +// not yet block. Flipping it to blocking is a one-line change at the caller, +// deliberately deferred until it has been green on a real release commit — +// a gate never green on its own target is not a gate (#3731). + +/// One surface's verdict about the subject, with the evidence it decided on. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct SurfaceVerdict { + /// The surface that decided, named as the user would invoke it. + pub surface: &'static str, + pub verdict: provable_contracts::ontology::verdict::Verdict, + /// What it measured. Printed verbatim in a contradiction. + pub detail: String, +} + +impl SurfaceVerdict { + pub fn new( + surface: &'static str, + verdict: provable_contracts::ontology::verdict::Verdict, + detail: impl Into, + ) -> Self { + Self { surface, verdict, detail: detail.into() } + } + + /// A surface that was never evaluated on this subject. Absence is + /// `Unknown(NotRun)`, never agreement (#3821 done_when 3). + pub fn not_run(surface: &'static str) -> Self { + use provable_contracts::ontology::verdict::{Reason, Verdict}; + Self::new(surface, Verdict::Unknown(Reason::NotRun), "not evaluated on this subject") + } +} + +/// The reconciled answer for one subject. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct SubjectReport { + /// The ONT-6 meet over every surface. `Fail < Unknown(_) < Pass`, so this + /// is `min` and a single Fail carries. + pub verdict: provable_contracts::ontology::verdict::Verdict, + /// Set iff at least one surface said Pass AND at least one said Fail. + /// Names every surface and its value — the point of the row is that a + /// disagreement is reported rather than silently minned away. + pub contradiction: Option, + /// Surfaces that never evaluated this subject, named. + pub not_run: Vec<&'static str>, +} + +/// Reconcile every surface's verdict about ONE subject. +/// +/// Pure and GPU-free, so the contradiction logic is testable on a host with no +/// CUDA — which is the only way this row could have been tested at all. +pub(crate) fn reconcile_gpu_correct(subject: &str, surfaces: &[SurfaceVerdict]) -> SubjectReport { + use provable_contracts::ontology::verdict::{Reason, Verdict}; + + if surfaces.is_empty() { + return SubjectReport { + verdict: Verdict::Unknown(Reason::NotRun), + contradiction: None, + not_run: Vec::new(), + }; + } + + let verdict = surfaces.iter().map(|s| s.verdict).fold(Verdict::Pass, Verdict::meet); + + let passed: Vec<&SurfaceVerdict> = + surfaces.iter().filter(|s| s.verdict == Verdict::Pass).collect(); + let failed: Vec<&SurfaceVerdict> = + surfaces.iter().filter(|s| s.verdict == Verdict::Fail).collect(); + + let contradiction = (!passed.is_empty() && !failed.is_empty()).then(|| { + let mut lines = vec![format!( + "CONTRADICTION on {subject}: GPU-correct is one subject and {} of {} surfaces \ + disagree about it.", + passed.len() + failed.len(), + surfaces.len() + )]; + for s in surfaces { + lines.push(format!(" - {}: {:?} — {}", s.surface, s.verdict, s.detail)); + } + lines.push( + " Both cannot be right about the same model on the same host in the same binary." + .to_string(), + ); + lines.join("\n") + }); + + let not_run = surfaces + .iter() + .filter(|s| s.verdict == Verdict::Unknown(Reason::NotRun)) + .map(|s| s.surface) + .collect(); + + SubjectReport { verdict, contradiction, not_run } +} + +/// The subject tuple, rendered. Keyed on the model's CONTENT, not its path — +/// two files with the same name are not the same subject, and the same bytes +/// under two names are. +pub(crate) fn gpu_correct_subject(model_sha256: &str, host: &str, apr_version: &str) -> String { + let short = model_sha256.get(..12).unwrap_or(model_sha256); + format!("GPU-correct(model {short}, host {host}, apr {apr_version})") +} + +/// The two surfaces' names, as a user would invoke them. +pub(crate) const F2_GUARD_SURFACE: &str = "apr run --gpu (F2 parity guard)"; +pub(crate) const GOLDEN_SURFACE: &str = "apr qa golden_output"; + +/// The F2 guard's answer as a surface verdict. The guard is a probe that +/// always runs to a decision, so it is never `Unknown`. +pub(crate) fn f2_guard_verdict(admits: bool) -> SurfaceVerdict { + use provable_contracts::ontology::verdict::Verdict; + if admits { + SurfaceVerdict::new(F2_GUARD_SURFACE, Verdict::Pass, "admitted: GPU first tokens match CPU") + } else { + SurfaceVerdict::new( + F2_GUARD_SURFACE, + Verdict::Fail, + "REFUSED: GPU output diverges from CPU (apr run --gpu would fall back / exit 14)", + ) + } +} + +/// The golden gate's wiring (#3821): reconcile the guard's verdict with the +/// leg's, on the same model and host, and return the contradiction text when +/// they disagree. `subject` is only evaluated then — it hashes the model, which +/// is seconds on a multi-GB GGUF and not worth paying when they agree. +/// +/// REPORT_ONLY: the caller prints this and the gate verdict is unchanged. +pub(crate) fn golden_contradiction( + guard_admits: bool, + leg: &crate::commands::qa::GpuGoldenLeg, + subject: impl FnOnce() -> String, +) -> Option { + use provable_contracts::ontology::verdict::Verdict; + let guard = f2_guard_verdict(guard_admits); + let golden = SurfaceVerdict::new(GOLDEN_SURFACE, leg.subject_verdict(), leg.subject_detail()); + let pair = [guard, golden]; + let says = |v: Verdict| pair.iter().any(|s| s.verdict == v); + if !(says(Verdict::Pass) && says(Verdict::Fail)) { + return None; + } + reconcile_gpu_correct(&subject(), &pair).contradiction +} + +/// The subject for a mapped GGUF on this host and this binary. +pub(crate) fn subject_of_bytes(model_bytes: &[u8]) -> String { + use sha2::{Digest, Sha256}; + let sha = format!("{:x}", Sha256::digest(model_bytes)); + let host = std::fs::read_to_string("/etc/hostname") + .ok() + .map(|h| h.trim().to_string()) + .filter(|h| !h.is_empty()) + .or_else(|| std::env::var("HOSTNAME").ok()) + .unwrap_or_else(|| "unknown-host".to_string()); + gpu_correct_subject(&sha, &host, env!("CARGO_PKG_VERSION")) +} + +// NOTE ON NAMING: `Makefile`'s coverage target passes `--skip gpu_`, and +// libtest matches that substring against the WHOLE test path. Every name below +// therefore avoids the sequence `gpu_` — otherwise these rows would compile, +// pass locally, and be invisible to the coverage gate. The module is named +// `subject_verdict_tests` for the same reason (#3839). +// The golden gate's own leg, expressed in the same lattice as the guard so the +// two can be compared at all. #3821: until this existed the two surfaces spoke +// different languages — one returned bool, the other a five-variant enum — and +// "compare them" had no meaning. +impl crate::commands::qa::GpuGoldenLeg { + /// This leg's verdict ABOUT THE SUBJECT, which is not the same as whether + /// the gate passed. A leg that never ran, errored, or ran out of budget + /// says nothing about `GPU-correct`; only `Passed` and `WrongAnswer` do. + pub(crate) fn subject_verdict(&self) -> provable_contracts::ontology::verdict::Verdict { + use provable_contracts::ontology::verdict::{Reason, Verdict}; + match self { + Self::Passed => Verdict::Pass, + Self::WrongAnswer(_) => Verdict::Fail, + // Attempted, but produced nothing judgeable. Not evidence either way. + Self::Errored(_) | Self::NotRun(_) => { + Verdict::Unknown(Reason::NotRun) + } + } + } + + /// What this leg measured, for a contradiction message. + pub(crate) fn subject_detail(&self) -> String { + match self { + Self::Passed => "golden patterns matched on the device".to_string(), + Self::WrongAnswer(why) => why.clone(), + Self::Errored(e) => format!("errored: {e}"), + Self::NotRun(why) => (*why).to_string(), + } + } +} + +#[cfg(test)] +mod subject_verdict_tests { + use super::{gpu_correct_subject, reconcile_gpu_correct, SurfaceVerdict}; + use provable_contracts::ontology::verdict::{Reason, Verdict}; + + const RUN: &str = "apr run --gpu (F2 parity guard)"; + const QA: &str = "apr qa golden_output"; + + fn subject() -> String { + gpu_correct_subject("d98cdcbd03e17ce47681435b5150e34c", "lambda", "0.69.1") + } + + /// The row's whole reason for existing. `meet` already yields Fail, so a + /// silent `min` would look like an ordinary failure — indistinguishable + /// from both surfaces agreeing the model is bad. A contradiction is a + /// different fact and must READ differently. + #[test] + fn a_pass_and_a_fail_on_one_subject_are_named_as_a_contradiction() { + let r = reconcile_gpu_correct( + &subject(), + &[ + SurfaceVerdict::new(RUN, Verdict::Fail, "diverges at position 1, cosine 0.4153"), + SurfaceVerdict::new(QA, Verdict::Pass, "3 golden test cases passed"), + ], + ); + assert_eq!(r.verdict, Verdict::Fail, "the meet still carries the Fail"); + let msg = r.contradiction.expect("a Pass beside a Fail is a contradiction"); + assert!(msg.contains("CONTRADICTION")); + assert!(msg.contains(RUN) && msg.contains(QA), "both surfaces must be named: {msg}"); + assert!( + msg.contains("0.4153") && msg.contains("3 golden test cases passed"), + "both VALUES must be named, not just the surfaces: {msg}" + ); + } + + /// Agreement is not a contradiction, in either direction. A row that fired + /// on agreement would be noise and would be turned off. + #[test] + fn surfaces_that_agree_are_not_reported_as_contradicting() { + for v in [Verdict::Pass, Verdict::Fail] { + let r = reconcile_gpu_correct( + &subject(), + &[SurfaceVerdict::new(RUN, v, "x"), SurfaceVerdict::new(QA, v, "y")], + ); + assert_eq!(r.verdict, v); + assert!(r.contradiction.is_none(), "agreement on {v:?} is not a contradiction"); + } + } + + /// #3821 done_when 3. One surface passing while the other never ran must + /// NOT read as a pass — that is how a gate nobody executed becomes + /// "green". The lattice gives Unknown, and the unrun surface is named. + #[test] + fn absence_is_not_agreement() { + let r = reconcile_gpu_correct( + &subject(), + &[SurfaceVerdict::new(RUN, Verdict::Pass, "admitted"), SurfaceVerdict::not_run(QA)], + ); + assert_eq!( + r.verdict, + Verdict::Unknown(Reason::NotRun), + "a Pass beside an unrun surface is Unknown, never Pass" + ); + assert!(!r.verdict.arm(), "Unknown must not arm"); + assert_eq!(r.not_run, vec![QA], "the unrun surface must be named"); + assert!(r.contradiction.is_none(), "not running is not disagreeing"); + } + + /// The meet is the ONT-6 lattice min over all three levels, and a Fail + /// beside an Unknown is still a Fail — the strongest claim wins. + #[test] + fn the_verdict_is_the_lattice_meet() { + let cases: &[(Verdict, Verdict, Verdict)] = &[ + (Verdict::Pass, Verdict::Pass, Verdict::Pass), + (Verdict::Pass, Verdict::Fail, Verdict::Fail), + (Verdict::Fail, Verdict::Unknown(Reason::NotRun), Verdict::Fail), + (Verdict::Pass, Verdict::Unknown(Reason::NotRun), Verdict::Unknown(Reason::NotRun)), + (Verdict::Fail, Verdict::Fail, Verdict::Fail), + ]; + let wrong: Vec = cases + .iter() + .filter_map(|(a, b, want)| { + let got = reconcile_gpu_correct( + &subject(), + &[SurfaceVerdict::new(RUN, *a, ""), SurfaceVerdict::new(QA, *b, "")], + ) + .verdict; + (got != *want).then(|| format!("\n - {a:?} meet {b:?}: expected {want:?}, got {got:?}")) + }) + .collect(); + assert!(wrong.is_empty(), "the meet is not the lattice min:{}", wrong.join("")); + } + + /// Nothing measured is Unknown, never Pass. An empty surface set is how a + /// subject nobody looked at would otherwise arrive as green. + #[test] + fn a_subject_no_surface_evaluated_is_unknown_not_pass() { + let r = reconcile_gpu_correct(&subject(), &[]); + assert_eq!(r.verdict, Verdict::Unknown(Reason::NotRun)); + assert!(!r.verdict.arm()); + assert!(r.contradiction.is_none()); + } + + /// The subject is keyed on the model's CONTENT. Two files with the same + /// name are not one subject; the same bytes under two names are. + #[test] + fn the_subject_is_keyed_on_content_and_host_and_version() { + let a = gpu_correct_subject("aaaaaaaaaaaabbbb", "lambda", "0.69.1"); + assert_eq!(a, gpu_correct_subject("aaaaaaaaaaaabbbb", "lambda", "0.69.1")); + assert_ne!(a, gpu_correct_subject("ccccccccccccbbbb", "lambda", "0.69.1")); + assert_ne!(a, gpu_correct_subject("aaaaaaaaaaaabbbb", "gx10", "0.69.1")); + assert_ne!(a, gpu_correct_subject("aaaaaaaaaaaabbbb", "lambda", "0.69.0")); + assert!(a.contains("aaaaaaaaaaaa"), "the sha must appear: {a}"); + assert!(!a.contains("bbbb"), "only the first 12 hex are printed: {a}"); + } + + /// Only `Passed` and `WrongAnswer` are evidence about the subject. The + /// other three mean the gate produced nothing judgeable, and a gate that + /// produced nothing must not read as agreement with whatever the guard + /// said — that is how an unrun surface becomes a green one. + #[test] + fn only_a_judged_answer_is_evidence_about_the_subject() { + use crate::commands::qa::GpuGoldenLeg; + let cases: Vec<(GpuGoldenLeg, Verdict)> = vec![ + (GpuGoldenLeg::Passed, Verdict::Pass), + (GpuGoldenLeg::WrongAnswer("gibberish".into()), Verdict::Fail), + (GpuGoldenLeg::Errored("CUDA init".into()), Verdict::Unknown(Reason::NotRun)), + (GpuGoldenLeg::NotRun("no device"), Verdict::Unknown(Reason::NotRun)), + ]; + let wrong: Vec = cases + .iter() + .filter_map(|(leg, want)| { + let got = leg.subject_verdict(); + (got != *want).then(|| format!("\n - {leg:?}: expected {want:?}, got {got:?}")) + }) + .collect(); + assert!(wrong.is_empty(), "leg-to-verdict mapping drifted:{}", wrong.join("")); + } + + /// The detail is what a contradiction message prints, so a leg that says + /// nothing useful makes the contradiction unreadable. + #[test] + fn every_leg_states_what_it_measured() { + use crate::commands::qa::GpuGoldenLeg; + for leg in [ + GpuGoldenLeg::Passed, + GpuGoldenLeg::WrongAnswer("fragment repeats".into()), + GpuGoldenLeg::Errored("init".into()), + GpuGoldenLeg::NotRun("no device"), + ] { + let d = leg.subject_detail(); + assert!(!d.trim().is_empty(), "{leg:?} produced an empty detail"); + } + assert!(GpuGoldenLeg::WrongAnswer("fragment repeats".into()) + .subject_detail() + .contains("fragment repeats"), "the reason must survive into the detail"); + } + + /// The golden wiring: every (guard, leg) pair, and whether it names a + /// contradiction. The dangerous row is guard-refuses + gate-passes, which + /// `meet` alone would report as an ordinary Fail. + #[test] + fn the_golden_wiring_names_exactly_the_disagreeing_pairs() { + use super::golden_contradiction; + use crate::commands::qa::GpuGoldenLeg; + let legs = || { + vec![ + GpuGoldenLeg::Passed, + GpuGoldenLeg::WrongAnswer("fragment repeats".into()), + GpuGoldenLeg::Errored("init".into()), + GpuGoldenLeg::NotRun("no device"), + ] + }; + let mut got = Vec::new(); + for admits in [true, false] { + for leg in legs() { + let c = golden_contradiction(admits, &leg, subject); + got.push((admits, format!("{leg:?}"), c.is_some())); + } + } + let named: Vec<_> = got.iter().filter(|r| r.2).map(|r| (r.0, r.1.clone())).collect(); + assert_eq!( + named, + vec![ + (true, format!("{:?}", GpuGoldenLeg::WrongAnswer("fragment repeats".into()))), + (false, format!("{:?}", GpuGoldenLeg::Passed)), + ], + "only guard-admits+wrong-answer and guard-refuses+passed contradict" + ); + let text = golden_contradiction(false, &GpuGoldenLeg::Passed, subject).expect("contradiction"); + assert!(text.contains("REFUSED") && text.contains(QA) && text.contains(RUN), "{text}"); + } + + /// The subject is hashed only when there is something to report. + #[test] + fn agreement_never_hashes_the_model() { + use super::golden_contradiction; + use crate::commands::qa::GpuGoldenLeg; + let c = golden_contradiction(true, &GpuGoldenLeg::Passed, || { + panic!("subject evaluated on agreement") + }); + assert!(c.is_none()); + } + + /// The subject names content, host and version. + #[test] + fn the_subject_of_bytes_is_the_content_digest() { + let s = super::subject_of_bytes(b"abc"); + // sha256("abc") = ba7816bf8f01... + assert!(s.contains("model ba7816bf8f01"), "{s}"); + assert!(s.contains(env!("CARGO_PKG_VERSION")), "{s}"); + } + + /// A short or empty sha must not panic on the 12-char slice. + #[test] + fn a_short_digest_does_not_panic() { + assert!(gpu_correct_subject("abc", "h", "v").contains("abc")); + assert!(gpu_correct_subject("", "h", "v").contains("host h")); + } +} diff --git a/crates/apr-cli/src/commands/gpu_isolation_result.rs b/crates/apr-cli/src/commands/gpu_isolation_result.rs index be0bc8b1fd..82a7b6b820 100644 --- a/crates/apr-cli/src/commands/gpu_isolation_result.rs +++ b/crates/apr-cli/src/commands/gpu_isolation_result.rs @@ -207,10 +207,10 @@ fn run_gpu_isolation_test(path: &Path) -> Result { QuantizedGenerateConfig, }; - let model_bytes = std::fs::read(path) - .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let gguf = GGUFModel::from_bytes(&model_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; + // #3750: one map; the tokenizer comes from its header, not from a whole-file read + let mapped = MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; + let gguf = &mapped.model; let bos = aprender::demo::SpecialTokens::qwen2().bos_id; let tokens_a = gguf @@ -227,8 +227,6 @@ fn run_gpu_isolation_test(path: &Path) -> Result { ..Default::default() }; - let mapped = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; let model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("Model failed: {e}")))?; let mut cuda_model = OwnedQuantizedModelCuda::new(model, 0) diff --git a/crates/apr-cli/src/commands/mod.rs b/crates/apr-cli/src/commands/mod.rs index 70fa582a57..42cb06892d 100644 --- a/crates/apr-cli/src/commands/mod.rs +++ b/crates/apr-cli/src/commands/mod.rs @@ -157,6 +157,7 @@ pub(crate) mod test_llm_band; // #2399: gated on the crate it actually needs (aprender-explain, aliased // `trueno-explain`) rather than on `full`, so `--features ptx` is enough and a // user does not have to pull CUDA + training to analyze a .ptx file. +pub(crate) mod model_header; #[cfg(feature = "trueno-explain")] pub(crate) mod ptx_explain; pub(crate) mod ptx_map; diff --git a/crates/apr-cli/src/commands/model_header.rs b/crates/apr-cli/src/commands/model_header.rs new file mode 100644 index 0000000000..cc2d96e07d --- /dev/null +++ b/crates/apr-cli/src/commands/model_header.rs @@ -0,0 +1,298 @@ +//! Header-only reads of a model file (#3750). +//! +//! `apr qa` asked "is this GGUF?" and "which architecture is it?" by reading the +//! WHOLE model into memory. On Qwen3-30B-A3B that is 17.3 GiB per read: three +//! reads in the capability gate alone, plus a copy. RSS swung past 30 GB, and on +//! GB10's unified memory that competes with the GPU and the CI containers (the +//! 2026-09-21 15:56Z OOM killed 18 of them). None of those questions needs +//! tensor data. Each needs a few bytes or the header, and these readers never +//! touch more. + +use std::io::Read; +use std::path::Path; + +use aprender::format::gguf::reader::GgufReader; +use aprender::format::v2::{AprV2Header, HEADER_SIZE_V2}; + +/// The first GGUF header prefix tried. Measured on 21 local GGUFs (2026-09-21): headers are +/// 5.7 MiB (Qwen2.5 / Qwen3) to 10.5 MiB (Qwen3.5, a 248,320-token vocabulary), on files up +/// to 17.3 GiB, so one read covers every model on the ladder. +const GGUF_HEADER_FIRST: usize = 16 << 20; + +/// The most a header read ever takes: about 24x the largest header measured. A file whose +/// header does not parse inside it is refused, never read whole. +const HEADER_CAP: usize = 256 << 20; + +/// At most the first `n` bytes of `path`. +pub(crate) fn read_prefix(path: &Path, n: usize) -> std::io::Result> { + let mut buf = Vec::with_capacity(n.min(1 << 20)); + std::fs::File::open(path)? + .take(n as u64) + .read_to_end(&mut buf)?; + Ok(buf) +} + +/// A GGUF header parsed from a growing prefix of the file: 16 MiB first, doubling, never +/// more than 256 MiB. +/// +/// The reader holds ONLY that prefix. Its metadata and tensor table are complete, and its +/// tensor data is absent, so it answers header questions (architecture, the tensor +/// `(name, type)` table, rope/context/eps) and nothing else. +pub(crate) fn gguf_header(path: &Path) -> Result { + gguf_header_within(path, GGUF_HEADER_FIRST, HEADER_CAP) +} + +/// [`gguf_header`] with its two limits as parameters, so the case table can exercise the +/// growth and the cap on small files. +fn gguf_header_within(path: &Path, first: usize, cap: usize) -> Result { + let len = std::fs::metadata(path) + .map_err(|e| format!("cannot stat {}: {e}", path.display()))? + .len(); + let mut n = first.min(cap); + loop { + let prefix = + read_prefix(path, n).map_err(|e| format!("cannot read {}: {e}", path.display()))?; + let whole_file = prefix.len() as u64 >= len; + match GgufReader::from_bytes(prefix) { + Ok(reader) => return Ok(reader), + Err(e) if whole_file => return Err(format!("GGUF header parse failed: {e}")), + Err(e) if n >= cap => { + return Err(format!( + "no complete GGUF header in the first {cap} bytes of {} ({e}); refused rather than read whole", + path.display() + )) + }, + Err(_) => n = (n * 2).min(cap), + } + } +} + +/// The architecture and the tensor `(name, GGML type)` table of a GGUF, from its header. +/// +/// `None` when the file is not a GGUF, the header does not parse, or it names no +/// architecture. These are the same values the whole-file read produced, from the same reader. +pub(crate) fn gguf_arch_and_tensors(path: &Path) -> Option<(String, Vec<(String, u32)>)> { + if read_prefix(path, 4).ok()?.as_slice() != b"GGUF" { + return None; + } + let reader = gguf_header(path).ok()?; + let arch = reader.architecture()?; + let tensors = reader + .tensors + .into_iter() + .map(|t| (t.name, t.dtype)) + .collect(); + Some((arch, tensors)) +} + +/// The bytes of an APR v2 file before its tensor data: the header, the metadata and the +/// tensor index, which is everything `AprV2Reader::from_bytes` parses. The header's own +/// `data_offset` bounds it, and so does `HEADER_CAP`. +pub(crate) fn apr_header_prefix(path: &Path) -> Result, String> { + let head = read_prefix(path, HEADER_SIZE_V2) + .map_err(|e| format!("cannot read {}: {e}", path.display()))?; + let header = + AprV2Header::from_bytes(&head).map_err(|e| format!("APR header parse failed: {e}"))?; + let n = usize::try_from(header.data_offset) + .ok() + .filter(|&n| n <= HEADER_CAP) + .ok_or_else(|| { + format!( + "APR data_offset {} is past the {} MiB header cap; refused rather than read whole", + header.data_offset, + HEADER_CAP >> 20 + ) + })?; + read_prefix(path, n.max(HEADER_SIZE_V2)) + .map_err(|e| format!("cannot read {}: {e}", path.display())) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Write; + + /// A real two-tensor GGUF (`llama`, F32), optionally with a large metadata string so the + /// header outgrows a small first read, then extended SPARSELY to `sparse_len` bytes: a + /// multi-GiB model file that costs no disk. + fn gguf_fixture(sparse_len: u64, pad_metadata: usize) -> tempfile::NamedTempFile { + use aprender::format::gguf::{export_tensors_to_gguf, GgmlType, GgufTensor, GgufValue}; + let file = tempfile::NamedTempFile::with_suffix(".gguf").expect("temp file"); + { + let mut w = std::io::BufWriter::new(file.as_file()); + let tensors = vec![ + GgufTensor { + name: "token_embd.weight".into(), + shape: vec![4, 8], + dtype: GgmlType::F32, + data: vec![0u8; 128], + }, + GgufTensor { + name: "blk.0.attn_q.weight".into(), + shape: vec![8, 8], + dtype: GgmlType::F32, + data: vec![0u8; 256], + }, + ]; + let metadata = vec![ + ( + "general.architecture".to_string(), + GgufValue::String("llama".into()), + ), + ( + "general.name".to_string(), + GgufValue::String("x".repeat(pad_metadata)), + ), + ("llama.block_count".to_string(), GgufValue::Uint32(1)), + ]; + export_tensors_to_gguf(&mut w, &tensors, &metadata).expect("write GGUF"); + w.flush().expect("flush"); + } + if sparse_len > 0 { + file.as_file().set_len(sparse_len).expect("extend sparsely"); + } + file + } + + #[test] + fn read_prefix_reads_at_most_n_bytes() { + let mut f = tempfile::NamedTempFile::new().expect("temp file"); + f.write_all(b"0123456789").expect("write"); + assert_eq!(read_prefix(f.path(), 4).expect("read"), b"0123"); + assert_eq!(read_prefix(f.path(), 100).expect("read"), b"0123456789"); + } + + #[test] + fn a_multi_gib_gguf_answers_from_its_header() { + let f = gguf_fixture(2 << 30, 0); + assert_eq!(std::fs::metadata(f.path()).expect("stat").len(), 2 << 30); + let (arch, tensors) = gguf_arch_and_tensors(f.path()).expect("the header parses"); + assert_eq!(arch, "llama"); + assert_eq!( + tensors, + vec![ + ("token_embd.weight".to_string(), 0), + ("blk.0.attn_q.weight".to_string(), 0) + ] + ); + } + + #[test] + fn a_header_larger_than_the_first_read_is_found_by_growing() { + let f = gguf_fixture(0, 5_000); + let reader = gguf_header_within(f.path(), 1024, 1 << 20).expect("grows past 1 KiB"); + assert_eq!(reader.architecture().as_deref(), Some("llama")); + } + + #[test] + fn a_header_past_the_cap_is_refused_never_read_whole() { + let f = gguf_fixture(1 << 30, 5_000); + let e = gguf_header_within(f.path(), 1024, 4096) + .expect_err("a 4 KiB cap cannot hold a 5 KB header"); + assert!(e.contains("refused rather than read whole"), "{e}"); + } + + #[test] + fn a_file_that_is_not_gguf_has_no_gguf_header() { + let mut f = tempfile::NamedTempFile::new().expect("temp file"); + f.write_all(b"APR\0 not a gguf").expect("write"); + assert!(gguf_arch_and_tensors(f.path()).is_none()); + } + + #[test] + fn an_apr_prefix_ends_where_tensor_data_starts_and_still_parses() { + use aprender::format::v2::{AprV2Metadata, AprV2Reader, AprV2Writer, TensorDType}; + let mut writer = AprV2Writer::new(AprV2Metadata::new("test")); + writer.add_tensor( + "w", + TensorDType::F32, + vec![256, 256], + vec![0u8; 256 * 256 * 4], + ); + let mut bytes = Vec::new(); + writer.write_to(&mut bytes).expect("write APR"); + let mut f = tempfile::NamedTempFile::with_suffix(".apr").expect("temp file"); + f.write_all(&bytes).expect("write"); + + let prefix = apr_header_prefix(f.path()).expect("the header, metadata and index"); + let header = AprV2Header::from_bytes(&prefix).expect("header"); + assert_eq!( + prefix.len() as u64, + header.data_offset, + "exactly up to the tensor data" + ); + assert!( + prefix.len() < bytes.len() / 100, + "{} of {} bytes", + prefix.len(), + bytes.len() + ); + let reader = AprV2Reader::from_bytes(&prefix).expect("AprV2Reader parses the prefix"); + assert_eq!(reader.tensor_names(), vec!["w"]); + } + + /// The done_when row: on a sparse 2 GiB GGUF, the header readers' PEAK RSS stays small. + /// Measured in a CHILD process (this test binary, re-run on the probe below), because + /// `cargo test` shares one process between tests and its peak would be theirs. The bound + /// is measured, not guessed: see `PEAK_RSS_BOUND_KB`. + #[cfg(target_os = "linux")] + #[test] + fn header_reads_of_a_2_gib_gguf_keep_peak_rss_small() { + let f = gguf_fixture(2 << 30, 0); + let out = std::process::Command::new(std::env::current_exe().expect("this test binary")) + .args([ + "--exact", + "commands::model_header::tests::peak_rss_probe", + "--nocapture", + "--test-threads=1", + ]) + .env(PROBE_ENV, f.path()) + .output() + .expect("run the probe"); + let text = String::from_utf8_lossy(&out.stdout); + let hwm = text + .lines() + .find_map(|l| l.split("PEAK_RSS_KB=").nth(1)) // libtest prints it after the test name + .and_then(|v| v.split_whitespace().next()?.parse::().ok()) + .unwrap_or_else(|| panic!("the probe reported no peak: {text}{}", String::from_utf8_lossy(&out.stderr))); + assert!( + hwm < PEAK_RSS_BOUND_KB, + "peak RSS {hwm} KiB reading the header of a 2 GiB GGUF (bound {PEAK_RSS_BOUND_KB} KiB): \ + a whole-file read is back" + ); + } + + const PROBE_ENV: &str = "APR_3750_PEAK_RSS_PROBE"; + + /// Measured on this commit (x86-64 debug test binary, 2026-09-21), the probe's peak RSS was: + /// 44.4 MiB on this 2 GiB sparse fixture; 59.5-60.3 MiB on a real 0.5B GGUF (5.7 MiB header); + /// 57.4-59.0 MiB on the real Qwen3-30B-A3B GGUF (17.3 GiB). With a whole-file `std::fs::read` + /// put back into `gguf_arch_and_tensors`, it was 2,136,776 KiB (2.04 GiB) on this fixture. The + /// bound sits about 4x above the header path and 8x below the regression. + const PEAK_RSS_BOUND_KB: u64 = 256 * 1024; + + /// Not a test on its own: with the probe variable unset it does nothing. The test above + /// re-runs this binary on it, in a fresh process, to read that process's peak RSS. + #[cfg(target_os = "linux")] + #[test] + fn peak_rss_probe() { + let Some(path) = std::env::var_os(PROBE_ENV) else { + return; + }; + let path = std::path::PathBuf::from(path); + assert!( + gguf_arch_and_tensors(&path).is_some(), + "the probe's fixture has a header" + ); + assert!(gguf_header(&path).is_ok()); + let _ = super::super::qa_capability::cpu_only_architecture(&path); + let _ = super::super::qa_capability::hybrid_loader_architecture(&path); + let status = std::fs::read_to_string("/proc/self/status").expect("/proc/self/status"); + let hwm = status + .lines() + .find_map(|l| l.strip_prefix("VmHWM:")) + .and_then(|v| v.trim().trim_end_matches("kB").trim().parse::().ok()) + .expect("VmHWM in /proc/self/status"); + println!("PEAK_RSS_KB={hwm}"); + } +} diff --git a/crates/apr-cli/src/commands/mono.rs b/crates/apr-cli/src/commands/mono.rs index f6c4664bcd..f3fe5758c8 100644 --- a/crates/apr-cli/src/commands/mono.rs +++ b/crates/apr-cli/src/commands/mono.rs @@ -9,6 +9,7 @@ //! Contract: contracts/publish-workspace-v1.yaml use crate::error::CliError; +use batuta_common::cli_roles::FreeText; use clap::Subcommand; use std::collections::BTreeMap; use std::io::Write; @@ -28,7 +29,7 @@ pub enum MonoCommands { Shims { /// Output directory for shim crates #[arg(long, default_value = "shims")] - output: String, + output: FreeText, }, /// Verify workspace invariants (FALSIFY-MONO-* and FALSIFY-BUILD-*) diff --git a/crates/apr-cli/src/commands/output_verification.rs b/crates/apr-cli/src/commands/output_verification.rs index adf367f376..537655780d 100644 --- a/crates/apr-cli/src/commands/output_verification.rs +++ b/crates/apr-cli/src/commands/output_verification.rs @@ -15,11 +15,11 @@ fn run_metadata_plausibility_gate(path: &Path, config: &QaConfig) -> Result Result = Vec::new(); let mut checks_passed = 0usize; @@ -38,7 +38,7 @@ fn run_metadata_plausibility_gate(path: &Path, config: &QaConfig) -> Result, Option, Option, Option); -/// Extract model metadata from file bytes (GGUF, APR, or SafeTensors format). -fn extract_model_metadata(data: &[u8], path: &Path) -> Result { - let magic = &data[0..4]; +/// Extract model metadata (GGUF, APR, or SafeTensors format) given the file's magic. +/// +/// #3750: each format is read to the end of its header and no further: the GGUF header +/// prefix, the APR header + metadata + tensor index, or SafeTensors' sibling config.json. +fn extract_model_metadata(magic: &[u8], path: &Path) -> Result { + let magic = &magic[0..4]; if magic == b"GGUF" { - // GGUF format: use GgufReader - let reader = aprender::format::gguf::reader::GgufReader::from_bytes(data.to_vec()) + // GGUF format: use GgufReader, over the header prefix + let reader = super::model_header::gguf_header(path) .map_err(|e| CliError::ValidationFailed(format!("GGUF parse failed: {e}")))?; let arch = reader.architecture(); let rope_theta = reader.rope_theta(); @@ -204,7 +207,9 @@ fn extract_model_metadata(data: &[u8], path: &Path) -> Result { } else if &magic[0..3] == b"APR" || magic == b"APRN" { // APR format: parse v2 header + JSON metadata use aprender::format::v2::AprV2Reader; - let reader = AprV2Reader::from_bytes(data) + let prefix = super::model_header::apr_header_prefix(path) + .map_err(|e| CliError::ValidationFailed(format!("APR parse failed: {e}")))?; + let reader = AprV2Reader::from_bytes(&prefix) .map_err(|e| CliError::ValidationFailed(format!("APR parse failed: {e}")))?; let meta = reader.metadata(); let _ = path; @@ -605,14 +610,30 @@ fn validate_gpu_golden_output( } let model = OwnedQuantizedModel::from_mapped(mapped) .map_err(|e| CliError::ValidationFailed(format!("Model failed: {e}")))?; - let generated = match OwnedQuantizedModelCuda::new(model, 0) { - Ok(mut cuda_model) => cuda_model - .generate_gpu_resident(prompt_tokens, gen_config) - .map(|gpu_tokens| gguf.decode(&gpu_tokens)) - .map_err(|e| format!("GPU generation: {e}")), - Err(e) => Err(format!("CUDA init on device 0: {e}")), + // #3821: consult the SAME F2 parity guard `apr run --gpu` does, on the same + // prompt, before generating — so the two surfaces that decide GPU-correct + // for this model are compared instead of forming separate opinions. + // generate_gpu_resident resets the KV cache, so the probe does not consume it. + let (guard_admits, generated) = match OwnedQuantizedModelCuda::new(model, 0) { + Ok(mut cuda_model) => { + let admits = + realizar::infer::gpu_parity_admits(&mut cuda_model, gen_config, prompt_tokens); + let generated = cuda_model + .generate_gpu_resident(prompt_tokens, gen_config) + .map(|gpu_tokens| gguf.decode(&gpu_tokens)) + .map_err(|e| format!("GPU generation: {e}")); + (Some(admits), generated) + }, + Err(e) => (None, Err(format!("CUDA init on device 0: {e}"))), }; - Ok(GpuGoldenLeg::judge(generated, expected_patterns)) + let leg = GpuGoldenLeg::judge(generated, expected_patterns); + // REPORT_ONLY (#3821's ruling): name a contradiction, do not change the verdict. + if let Some(admits) = guard_admits { + if let Some(text) = golden_contradiction(admits, &leg, || subject_of_bytes(mapped.data())) { + eprintln!("{}", text.red()); + } + } + Ok(leg) } /// Note, in a verbose human-readable run, that the GPU half of the golden gate diff --git a/crates/apr-cli/src/commands/parity.rs b/crates/apr-cli/src/commands/parity.rs index 677f56e98b..10c6a78f61 100644 --- a/crates/apr-cli/src/commands/parity.rs +++ b/crates/apr-cli/src/commands/parity.rs @@ -476,3 +476,4 @@ include!("parity_diagnose_tests.rs"); include!("parity_refusal_tests.rs"); include!("parity_hybrid.rs"); include!("parity_hybrid_tests.rs"); +include!("parity_moe.rs"); diff --git a/crates/apr-cli/src/commands/parity_03.rs b/crates/apr-cli/src/commands/parity_03.rs index 571152634b..d221042316 100644 --- a/crates/apr-cli/src/commands/parity_03.rs +++ b/crates/apr-cli/src/commands/parity_03.rs @@ -124,8 +124,11 @@ pub fn run(file: &Path, prompt: &str, _assert: bool, verbose: bool, json: bool) // HAS both forwards — but it is not the dense pair. `OwnedQuantizedModel:: // from_mapped` below would hand it straight to `dense_loader_refusal`, // which is the FAIL this dispatch replaces with a measurement. - if parity_arm(mapped.model.architecture().unwrap_or_default()) == ParityArm::Hybrid { - return run_hybrid(file, &mapped, &tokens, verbose, json); + match parity_arm(mapped.model.architecture().unwrap_or_default()) { + ParityArm::Hybrid => return run_hybrid(file, &mapped, &tokens, verbose, json), + // #3714 R2: the routed-expert pair — the one the runtime F2 guard judges. + ParityArm::Moe => return run_moe(file, &mapped, &tokens, verbose, json), + ParityArm::Dense => {}, } let model = OwnedQuantizedModel::from_mapped(&mapped) diff --git a/crates/apr-cli/src/commands/parity_hybrid.rs b/crates/apr-cli/src/commands/parity_hybrid.rs index db5ccc6ac0..180b7f3750 100644 --- a/crates/apr-cli/src/commands/parity_hybrid.rs +++ b/crates/apr-cli/src/commands/parity_hybrid.rs @@ -18,6 +18,9 @@ pub(crate) enum ParityArm { /// `Qwen35Model::forward_single_qwen35` vs `Qwen35CudaModel::forward_single`. Hybrid, + /// `OwnedQuantizedModel::forward_single_qwen3_moe_with_cache` (FP32 + /// activations) vs `Qwen3MoeCudaModel::forward_single` (#3714 R2). + Moe, /// `OwnedQuantizedModel::forward_single_with_cache` vs `forward_gpu_resident`. Dense, } @@ -34,6 +37,8 @@ pub(crate) enum ParityArm { pub(crate) fn parity_arm(architecture: &str) -> ParityArm { if realizar::gguf::hybrid_forward_handles(architecture) { ParityArm::Hybrid + } else if realizar::gguf::moe_forward_handles(architecture) { + ParityArm::Moe } else { ParityArm::Dense } diff --git a/crates/apr-cli/src/commands/parity_hybrid_tests.rs b/crates/apr-cli/src/commands/parity_hybrid_tests.rs index 4b1d282aac..a8fdc96d37 100644 --- a/crates/apr-cli/src/commands/parity_hybrid_tests.rs +++ b/crates/apr-cli/src/commands/parity_hybrid_tests.rs @@ -19,6 +19,21 @@ mod parity_arm_tests { ); } + /// #3714 R2: the spellings the runtime dispatches to the routed-expert + /// forward take the MoE arm; `qwen35moe` (the GGUF Qwen3.5-MoE tag) reaches + /// no MoE forward and does not. + #[test] + fn qwen3moe_is_measured_by_the_moe_arm() { + for arch in ["qwen3moe", "qwen3_moe"] { + assert_eq!( + parity_arm(arch), + ParityArm::Moe, + "{arch}: both routed-expert forwards exist (#3367 CPU, #3714 CUDA)" + ); + } + assert_ne!(parity_arm("qwen35moe"), ParityArm::Moe); + } + #[test] fn every_other_architecture_keeps_the_dense_arm() { for arch in ["llama", "qwen2", "qwen3", "gemma", "phi3", ""] { diff --git a/crates/apr-cli/src/commands/parity_moe.rs b/crates/apr-cli/src/commands/parity_moe.rs new file mode 100644 index 0000000000..d7e9441ed3 --- /dev/null +++ b/crates/apr-cli/src/commands/parity_moe.rs @@ -0,0 +1,231 @@ +// ═══════════════════════════════════════════════════════════════════════════════ +// #3714 R2: THE MoE ARM — Qwen3-MoE CPU vs CUDA +// ═══════════════════════════════════════════════════════════════════════════════ +// +// `apr parity` refused qwen3moe from the header (PMAT-1098, #3367): the dense +// loop would read the MoE placeholder, and no GPU forward existed. #3714 R1 +// built one (`Qwen3MoeCudaModel`), so the architecture the runtime dispatches +// to the routed-expert forward has BOTH forwards and parity measures them. +// +// The pair is the one the runtime F2 guard judges (`qwen3_moe_dispatch`): +// the CPU `forward_single_qwen3_moe_with_cache` with FP32 activations against +// `Qwen3MoeCudaModel::forward_single`. The GPU runs float GEMVs; judged against +// the default Q8_K-activation CPU it lost 0.13 cosine on Qwen3-30B-A3B- +// Instruct-2507's first tokens to the REFERENCE's quantization (#3714, #3751), +// so parity measuring any other pair would report a divergence the runtime +// never sees — or miss one it rejects. + +/// What one MoE measurement produced: the per-position metrics, plus the +/// attention geometry `auto_diagnose` prints. +#[cfg(feature = "cuda")] +struct MoeParity { + metrics: Vec, + hidden_dim: usize, + num_heads: usize, + kv_heads: usize, +} + +/// Measure the Qwen3-MoE routed-expert forward: CPU (FP32 activations) +/// against its CUDA twin, position by position over `tokens`, into the same +/// `SpcMetrics` the dense and hybrid loops produce. +/// +/// Both backends see the SAME token at the SAME position from their own fresh +/// state, as the F2 guard does, so a divergence this reports is one that would +/// reject the GPU at inference time. +/// +/// # Errors +/// MoE tensors that will not load, a CUDA device that will not initialize, a +/// device that cannot hold the model (the capacity plan's arithmetic, exit 14 — +/// a busy or small GPU is not a parity result), or either forward failing. +#[cfg(feature = "cuda")] +fn measure_moe( + mapped: &realizar::gguf::MappedGGUFModel, + tokens: &[u32], + verbose: bool, +) -> Result { + use realizar::gguf::{OwnedQuantizedKVCache, OwnedQuantizedModel, Qwen3MoeCudaModel}; + + let model = OwnedQuantizedModel::from_mapped(mapped) + .map_err(|e| CliError::ValidationFailed(format!("Failed to load the MoE model: {e}")))?; + let shape = realizar::infer::qwen3_moe_dispatch::qwen3_moe_shape(mapped) + .map_err(CliError::ValidationFailed)?; + let config = model.config(); + let layers = (0..config.num_layers) + .map(|il| { + realizar::gguf::qwen3_moe_load::load_qwen3_moe_layer(&mapped.model, mapped.data(), il) + }) + .collect::, _>>() + .map_err(|e| CliError::ValidationFailed(format!("Failed to load the MoE layers: {e}")))?; + + let hidden_dim = config.hidden_dim; + let num_heads = config.num_heads; + let kv_heads = config.num_kv_heads; + let (head_dim, _kv_dim, gqa_ratio) = head_geometry(hidden_dim, num_heads, kv_heads); + eprintln!( + " {} hidden={} heads={} kv_heads={} head_dim={} GQA={} layers={} vocab={}", + "Arch:".white().bold(), + hidden_dim, + num_heads, + kv_heads, + head_dim, + gqa_ratio, + config.num_layers, + config.vocab_size, + ); + eprintln!( + " {}", + format!( + "MoE: {} experts, {} routed per token, expert FFN {} (#3367 CPU / #3714 CUDA); \ + CPU reference with FP32 activations", + shape.num_experts, shape.num_experts_per_tok, shape.expert_dim + ) + .dimmed() + ); + + let executor = realizar::cuda::CudaExecutor::new(0) + .map_err(|e| CliError::ValidationFailed(format!("CUDA init failed: {e}")))?; + let max_seq = tokens.len() + 1; + let mut gpu = Qwen3MoeCudaModel::with_max_seq_len( + &model, + &layers, + shape, + mapped.data(), + executor, + max_seq, + ) + .map_err(|e| match e { + realizar::error::RealizarError::CapacityRefused(_) => { + CliError::BackendUnavailable(format!("{e}")) + }, + other => CliError::ValidationFailed(format!("CUDA MoE model build failed: {other}")), + })?; + let (device_name, vram_mb) = gpu.device_summary(); + eprintln!( + " {} {} ({} MB VRAM)", + "GPU:".white().bold(), + device_name.green(), + vram_mb, + ); + + let cpu_logits: Vec> = realizar::quantize::with_fp32_activations(|| { + let mut cache = OwnedQuantizedKVCache::from_config(model.config(), max_seq); + tokens + .iter() + .enumerate() + .map(|(pos, &token_id)| { + model + .forward_single_qwen3_moe_with_cache( + token_id, + &mut cache, + pos, + &layers, + shape.num_experts, + shape.num_experts_per_tok, + shape.expert_dim, + mapped.data(), + ) + .map_err(|e| format!("CPU forward failed at pos {pos}: {e}")) + }) + .collect::, String>>() + }) + .map_err(CliError::InferenceFailed)?; + + let mut gpu_state = gpu.new_state().map_err(|e| { + CliError::ValidationFailed(format!("CUDA decode state allocation failed: {e}")) + })?; + + eprintln!(); + print_header(); + + let mut metrics = Vec::with_capacity(tokens.len()); + for (pos, (&token_id, cpu)) in tokens.iter().zip(&cpu_logits).enumerate() { + let gpu_logits = gpu.forward_single(token_id, &mut gpu_state, pos).map_err(|e| { + CliError::InferenceFailed(format!("GPU forward failed at pos {pos}: {e}")) + })?; + let m = compute_metrics(cpu, &gpu_logits, pos, token_id); + print_row(&m); + if verbose && m.verdict().is_fail() { + print_verbose_failure(&m); + } + metrics.push(m); + } + + Ok(MoeParity { + metrics, + hidden_dim, + num_heads, + kv_heads, + }) +} + +/// The MoE arm of `run`: measure, then emit the shared report. +#[cfg(feature = "cuda")] +fn run_moe( + file: &Path, + mapped: &realizar::gguf::MappedGGUFModel, + tokens: &[u32], + verbose: bool, + json: bool, +) -> Result<()> { + let measured = measure_moe(mapped, tokens, verbose)?; + emit_parity_outcome( + file, + &measured.metrics, + measured.hidden_dim, + measured.num_heads, + measured.kv_heads, + json, + ) +} + +/// End-to-end on the real file (#3714 done_when 2): ≥ 64 positions, every +/// position PASSing parity's own verdict. Skips without the file, a device, or +/// room on it. +#[cfg(all(test, feature = "cuda"))] +mod parity_moe_tests { + use super::measure_moe; + + const MODEL: &str = "/home/noah/models/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf"; + + const PROMPT: &str = "<|im_start|>user\nThe history of the printing press begins in the \ + fifteenth century, when Johannes Gutenberg combined movable metal type, oil-based ink \ + and a wooden screw press into a system that could reproduce books quickly and cheaply. \ + Within fifty years, presses operated in more than two hundred cities across Europe. \ + Printed pamphlets carried arguments about religion, science and politics to readers who \ + had never owned a manuscript, and the price of a book fell by an order of magnitude."; + + #[test] + fn parity_moe_holds_at_64_positions_on_the_real_file() { + let path = std::env::var("APR_QWEN3MOE_GGUF").unwrap_or_else(|_| MODEL.to_string()); + if !std::path::Path::new(&path).exists() { + eprintln!("SKIP: {path} is absent"); + return; + } + if realizar::cuda::CudaExecutor::num_devices() == 0 { + eprintln!("SKIP: no CUDA device"); + return; + } + let mapped = realizar::gguf::MappedGGUFModel::from_path(&path).expect("map the GGUF"); + let tokens = mapped.model.encode(PROMPT).expect("tokenize"); + assert!(tokens.len() >= 64, "the prompt must reach 64 positions, got {}", tokens.len()); + let measured = match measure_moe(&mapped, &tokens, false) { + Ok(m) => m, + Err(crate::error::CliError::BackendUnavailable(why)) => { + eprintln!("SKIP: {why}"); + return; + }, + Err(e) => panic!("measure: {e}"), + }; + assert_eq!(measured.metrics.len(), tokens.len()); + for m in &measured.metrics { + assert!( + !m.verdict().is_fail(), + "pos {}: parity FAIL (cpu argmax {} gpu argmax {}, max |diff| {})", + m.position, + m.cpu_argmax, + m.gpu_argmax, + m.max_abs_diff + ); + } + } +} diff --git a/crates/apr-cli/src/commands/parity_per_op.rs b/crates/apr-cli/src/commands/parity_per_op.rs index 4b3b507fdd..b9f0ac76ab 100644 --- a/crates/apr-cli/src/commands/parity_per_op.rs +++ b/crates/apr-cli/src/commands/parity_per_op.rs @@ -37,6 +37,9 @@ pub fn run( let mapped = MappedGGUFModel::from_path(file) .map_err(|e| CliError::ValidationFailed(format!("GGUF load failed: {e}")))?; + if let Some(why) = per_op_refusal(mapped.model.architecture().unwrap_or_default()) { + return Err(CliError::NotImplemented(why)); + } let tokens = mapped.model.encode(prompt).unwrap_or_else(|| vec![1u32]); let model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("model load failed: {e}")))?; @@ -188,6 +191,50 @@ fn stage_files(dir: &Path) -> Vec<(String, Option)> { out } +/// #3714 R2: the per-op tap is wired into the DENSE pair only +/// (`forward_single_with_cache` vs `forward_gpu_resident`). A hybrid or +/// routed-expert file would reach the dense loop. For a MoE file that loop reads +/// the 0-byte FFN placeholders `from_gguf_for_moe` leaves, which reports a TOOL +/// gap as a divergence of the MODEL. So those arms are refused (exit 12, +/// `PARITY_REFUSED_EXIT`), naming the command that does measure them. `None` +/// for the dense arm. +#[cfg(feature = "inference")] +#[allow(dead_code)] // the only non-test caller is the cuda-gated `run` +pub(crate) fn per_op_refusal(architecture: &str) -> Option { + use super::parity::{parity_arm, ParityArm}; + let arm = match parity_arm(architecture) { + ParityArm::Dense => return None, + ParityArm::Hybrid => "hybrid (Qwen3.5)", + ParityArm::Moe => "routed-expert (qwen3moe)", + }; + Some(format!( + "apr parity --per-op: architecture '{architecture}' runs the {arm} forward, which has no \ + per-op tap; the per-op instrument is dense-only. `apr parity` without --per-op measures \ + this pair (logits at every position)." + )) +} + +#[cfg(all(test, feature = "inference"))] +mod per_op_refusal_tests { + use super::per_op_refusal; + + /// Every arm that is not the dense pair is refused; the dense one is not. + #[test] + fn per_op_refuses_the_arms_it_has_no_tap_for() { + for arch in ["qwen2", "qwen3", "llama"] { + assert_eq!(per_op_refusal(arch), None, "{arch} is the dense pair"); + } + for (arch, arm) in [("qwen35", "hybrid"), ("qwen3moe", "routed-expert")] { + let why = per_op_refusal(arch).unwrap_or_default(); + assert!(why.contains(arm), "{arch}: {why}"); + assert!( + why.contains("without --per-op"), + "{arch} must name what does measure it: {why}" + ); + } + } +} + #[cfg(not(feature = "cuda"))] pub fn run( _file: &Path, diff --git a/crates/apr-cli/src/commands/parity_refusal.rs b/crates/apr-cli/src/commands/parity_refusal.rs index 7110fac776..ea107006b6 100644 --- a/crates/apr-cli/src/commands/parity_refusal.rs +++ b/crates/apr-cli/src/commands/parity_refusal.rs @@ -151,6 +151,16 @@ pub(crate) fn parity_refusal_for<'n>( ) -> Option { use realizar::gguf::{unsupported_architecture_reason, ArchConstraints}; + // #3714 R2: the spellings the runtime dispatches to the routed-expert + // forward have BOTH forwards — CPU (#3367) and CUDA (`Qwen3MoeCudaModel`, + // #3714) — so parity measures them (the MoE arm) and must stop refusing + // them. `moe_forward_handles` is realizar's own dispatch predicate, the + // same one `apr run` and `apr qa` ask. Every OTHER spelling `is_moe` + // matches reaches no MoE forward and keeps the refusal below. + if realizar::gguf::moe_forward_handles(architecture) { + return None; + } + if ArchConstraints::from_architecture(architecture).is_moe { // `normalize_architecture` is not total over the MoE spellings // `ArchConstraints` accepts (`qwen3_5moe` hits its `_ => "llama"` diff --git a/crates/apr-cli/src/commands/parity_refusal_tests.rs b/crates/apr-cli/src/commands/parity_refusal_tests.rs index 6c7cdaac33..3fb9ea4b87 100644 --- a/crates/apr-cli/src/commands/parity_refusal_tests.rs +++ b/crates/apr-cli/src/commands/parity_refusal_tests.rs @@ -7,16 +7,26 @@ mod parity_refusal_case_table { use super::{capability_refusal, cuda_init_error, parity_refusal_for, PARITY_REFUSED_EXIT}; - /// The three architectures the 0.68.0 T-2 dogfood measured as FAIL rows. - /// Each one is a TOOL refusal, not a model defect. + /// The three architectures the 0.68.0 T-2 dogfood measured as FAIL rows + /// were TOOL refusals, not model defects. #3714 R2 lifts the MoE one for + /// the spellings the runtime dispatches to the routed-expert forward + /// (`moe_forward_handles`): they now have a CUDA forward as well as the CPU + /// one, and the MoE arm measures them. `qwen3_5_moe` is admitted with them + /// because the runtime's normalizer folds it to `qwen3_moe`, so `apr run` + /// would run the Qwen3-MoE forward on it too (it is the HF `model_type`; + /// GGUF Qwen3.5-MoE files carry `qwen35moe`). Every other spelling + /// `ArchConstraints` calls MoE reaches no MoE forward and keeps the refusal. #[test] fn parity_refusal_moe_architecture_is_refused_with_issue_3367() { - // `qwen3moe` is the RAW general.architecture string Qwen3-Coder-30B-A3B - // and Qwen3-30B-A3B ship; `qwen3_moe` is the canonical key. Every - // spelling `ArchConstraints` calls MoE must be refused, canonical or not. - for raw in ["qwen3moe", "qwen3_moe", "qwen3_5_moe", "qwen3_5moe"] { + for raw in ["qwen3moe", "qwen3_moe", "qwen3_5_moe"] { + assert!( + parity_refusal_for(raw, Vec::<&str>::new()).is_none(), + "{raw} runs the routed-expert forward on both backends: the MoE arm measures it" + ); + } + for raw in ["qwen3_5moe"] { let r = parity_refusal_for(raw, Vec::<&str>::new()).unwrap_or_else(|| { - panic!("{raw} must be refused: the dense parity loop cannot route it") + panic!("{raw} must be refused: no MoE forward handles it") }); assert_eq!(r.issue, "#3367"); assert!( @@ -37,12 +47,10 @@ mod parity_refusal_case_table { /// fold would do. When in doubt the refusal reports what the file said. #[test] fn parity_refusal_moe_names_the_canonical_key_only_when_the_normalizer_knows_it() { - for (raw, named) in [ - ("qwen3moe", "qwen3_moe"), - ("qwen3_moe", "qwen3_moe"), - ("qwen3_5_moe", "qwen3_moe"), - ("qwen3_5moe", "qwen3_5moe"), - ] { + // The spellings `normalize_architecture` folds to `qwen3_moe` are no + // longer refused (#3714 R2); the one it does not fold still is, under + // the raw tag. + for (raw, named) in [("qwen3_5moe", "qwen3_5moe")] { let r = parity_refusal_for(raw, Vec::<&str>::new()).expect("refused"); assert_eq!(r.architecture, named, "arch named for raw tag {raw}"); assert_ne!( @@ -127,7 +135,7 @@ mod parity_refusal_case_table { #[test] fn parity_refusal_exit_code_is_distinct_from_the_codes_parity_already_emits() { use crate::error::CliError; - let refused = parity_refusal_for("qwen3moe", Vec::<&str>::new()) + let refused = parity_refusal_for("qwen3_5moe", Vec::<&str>::new()) .expect("refused") .into_error(); assert_eq!(refused.exit_code_value(), PARITY_REFUSED_EXIT); @@ -150,10 +158,10 @@ mod parity_refusal_case_table { /// so the literal prefix is asserted here and in the script's case table. #[test] fn parity_refusal_line_has_the_shape_c14_greps_for() { - let r = parity_refusal_for("qwen3moe", Vec::<&str>::new()).expect("refused"); + let r = parity_refusal_for("qwen3_5moe", Vec::<&str>::new()).expect("refused"); let line = r.line(); assert!( - line.starts_with("parity: REFUSED architecture=qwen3_moe — "), + line.starts_with("parity: REFUSED architecture=qwen3_5moe — "), "C14 greps this prefix: {line}" ); assert!( diff --git a/crates/apr-cli/src/commands/pretrain.rs b/crates/apr-cli/src/commands/pretrain.rs index f7dac32a45..627087bdb0 100644 --- a/crates/apr-cli/src/commands/pretrain.rs +++ b/crates/apr-cli/src/commands/pretrain.rs @@ -1832,6 +1832,7 @@ mod tests { .expect("spawn parse thread") .join() .expect("parse thread must not panic") + .into() } #[test] @@ -1900,6 +1901,7 @@ mod tests { .expect("spawn parse thread") .join() .expect("parse thread must not panic") + .map(Into::into) } /// FALSIFY-APR-PRETRAIN-INIT-001: --init flag exists in clap surface. diff --git a/crates/apr-cli/src/commands/profile_ollama.rs b/crates/apr-cli/src/commands/profile_ollama.rs index 21f61ef7c2..666d4f22f6 100644 --- a/crates/apr-cli/src/commands/profile_ollama.rs +++ b/crates/apr-cli/src/commands/profile_ollama.rs @@ -404,8 +404,7 @@ fn safe_ratio(num: f64, denom: f64) -> f64 { /// Printed in place of a memory-vs-compute verdict while the arithmetic /// intensity is not measured. A magnitude may be reported; a verdict decided by -/// a number nothing measured may not. -#[cfg(feature = "inference")] +/// a number nothing measured may not. Not feature-gated: `print_roofline_section` reads it in every build (#4041). pub(crate) const ROOFLINE_AI_UNMEASURED: &str = "UNMEASURED"; /// Does the arithmetic intensity actually depend on the workload? diff --git a/crates/apr-cli/src/commands/qa.rs b/crates/apr-cli/src/commands/qa.rs index afe274ba62..9bdb06e302 100644 --- a/crates/apr-cli/src/commands/qa.rs +++ b/crates/apr-cli/src/commands/qa.rs @@ -467,6 +467,14 @@ pub fn run( } /// Dispatch a single QA gate: skip if flagged, otherwise run, then print and collect. +/// +/// A gate whose runner returns `Err` is recorded as a FAILED row carrying the +/// error, and the remaining gates still run (#3714 done_when 3). It used to be +/// `runner()?`, which abandoned `run_qa` before the report existed: on a +/// qwen3moe file the golden gate's dense CPU path errored and `apr qa --json` +/// printed ZERO bytes and exited 5 — every gate that had already passed, and +/// the one that failed, were lost. An error is a FAIL, never a skip and never +/// a pass; the report is always written. fn dispatch_gate( gates: &mut Vec, json: bool, @@ -478,7 +486,16 @@ fn dispatch_gate( let result = if skip { GateResult::skipped(name, skip_reason) } else { - runner()? + let start = Instant::now(); + runner().unwrap_or_else(|e| { + GateResult::failed( + name, + &format!("gate errored before it could judge the model: {e}"), + None, + None, + start.elapsed(), + ) + }) }; if !json { print_gate_result(&result); @@ -558,6 +575,7 @@ fn print_qa_summary(gates: &[GateResult], passed: bool, total_duration: Duration include!("qa_gguf.rs"); include!("output_verification.rs"); +include!("gpu_correct_subject.rs"); include!("golden_output.rs"); include!("speedup.rs"); include!("forward_error.rs"); diff --git a/crates/apr-cli/src/commands/qa_capability.rs b/crates/apr-cli/src/commands/qa_capability.rs index 2b66110a7e..c584fa77be 100644 --- a/crates/apr-cli/src/commands/qa_capability.rs +++ b/crates/apr-cli/src/commands/qa_capability.rs @@ -42,8 +42,8 @@ pub fn run_capability_gate(path: &Path, config: &QaConfig) -> Result ); } - // Read file header to detect format - let data = match std::fs::read(path) { + // #3750: the magic here and the header below, never the whole model (17.3 GiB on the 30B) + let magic = match super::model_header::read_prefix(path, 4) { Ok(d) => d, Err(e) => { let duration = start.elapsed(); @@ -57,7 +57,7 @@ pub fn run_capability_gate(path: &Path, config: &QaConfig) -> Result } }; - if data.len() < 4 { + if magic.len() < 4 { let duration = start.elapsed(); return Ok(GateResult::failed( "capability_match", @@ -69,8 +69,7 @@ pub fn run_capability_gate(path: &Path, config: &QaConfig) -> Result } // Only check GGUF files — APR/SafeTensors don't carry arch constraints yet - let magic = &data[0..4]; - if magic != b"GGUF" { + if magic.as_slice() != b"GGUF" { let duration = start.elapsed(); return Ok(GateResult::passed( "capability_match", @@ -82,7 +81,7 @@ pub fn run_capability_gate(path: &Path, config: &QaConfig) -> Result } // Parse GGUF to get architecture string and tensor (name, GGML type) pairs - let Some((arch, tensors)) = extract_gguf_arch_and_tensors(&data) else { + let Some((arch, tensors)) = super::model_header::gguf_arch_and_tensors(path) else { let duration = start.elapsed(); return Ok(GateResult::passed( "capability_match", @@ -374,13 +373,8 @@ pub(crate) fn cpu_only_architecture(path: &Path) -> bool { if !cfg!(feature = "cuda") { return false; } - let Ok(data) = std::fs::read(path) else { - return false; - }; - if data.len() < 4 || &data[0..4] != b"GGUF" { - return false; - } - let Some((arch, tensors)) = extract_gguf_arch_and_tensors(&data) else { + // #3750: the header, never the whole model + let Some((arch, tensors)) = super::model_header::gguf_arch_and_tensors(path) else { return false; }; is_cpu_only_architecture(&arch, tensors.iter().map(|(n, _)| n.as_str())) @@ -403,13 +397,8 @@ pub(crate) fn cpu_only_architecture(_path: &Path) -> bool { /// measured on some other model shape. #[cfg(feature = "inference")] pub(crate) fn hybrid_loader_architecture(path: &Path) -> bool { - let Ok(data) = std::fs::read(path) else { - return false; - }; - if data.len() < 4 || &data[0..4] != b"GGUF" { - return false; - } - let Some((arch, _)) = extract_gguf_arch_and_tensors(&data) else { + // #3750: the header, never the whole model + let Some((arch, _)) = super::model_header::gguf_arch_and_tensors(path) else { return false; }; cpu_forward_handles(&arch) @@ -421,21 +410,26 @@ pub(crate) fn hybrid_loader_architecture(_path: &Path) -> bool { false } -/// Extract the architecture string and the tensor table from GGUF metadata. +/// #3714: is this a Qwen3-MoE file? The dense loader BUILDS it (the MoE +/// placeholder fills its dense FFN) but cannot FORWARD it, so its golden and +/// throughput gates go through the runtime entry point — the routed-expert +/// forward, CUDA (#3714) or CPU (#3367) — and every gate measured through the +/// dense `OwnedQuantizedModel`/`OwnedQuantizedModelCuda` skips, saying so. /// -/// Uses aprender's GGUF reader to parse metadata without loading tensors. -/// #3477: the GGML type travels with the name, because the hybrid GPU quant -/// gate judges `(name, type)` pairs — reading the names alone was what let an -/// unsupported-quant DeltaNet tensor through. -fn extract_gguf_arch_and_tensors(data: &[u8]) -> Option<(String, Vec<(String, u32)>)> { - let reader = aprender::format::gguf::reader::GgufReader::from_bytes(data.to_vec()).ok()?; - let arch = reader.architecture()?; - let tensors = reader - .tensors - .into_iter() - .map(|t| (t.name, t.dtype)) - .collect(); - Some((arch, tensors)) +/// Reads ONLY the GGUF header, through the one bounded-prefix header reader +/// (`model_header::gguf_arch_and_tensors`, #3750). Not a map: realizar's +/// `MappedGGUFModel::from_path` maps with MAP_POPULATE + mlock, which faults +/// in the whole 18.5 GB file to answer a one-string question (#3761). +#[cfg(feature = "inference")] +pub(crate) fn moe_loader_architecture(path: &Path) -> bool { + super::model_header::gguf_arch_and_tensors(path) + .is_some_and(|(a, _)| realizar::gguf::moe_forward_handles(&a)) +} + +/// Without `inference` there is no runtime to route to. +#[cfg(not(feature = "inference"))] +pub(crate) fn moe_loader_architecture(_path: &Path) -> bool { + false } #[cfg(all(test, feature = "inference"))] @@ -666,3 +660,78 @@ mod qa_capability_ssm_tests { } } } + +#[cfg(all(test, feature = "inference"))] +mod moe_loader_architecture_tests { + use super::moe_loader_architecture; + + fn gguf_str(out: &mut Vec, s: &str) { + out.extend_from_slice(&(s.len() as u64).to_le_bytes()); + out.extend_from_slice(s.as_bytes()); + } + + /// A GGUF v3 header whose architecture key comes AFTER a string array and + /// a scalar. The file ENDS right after the header: no tensor infos, no data. + fn header_with_arch_last(arch: &str) -> Vec { + let mut h = b"GGUF".to_vec(); + h.extend_from_slice(&3u32.to_le_bytes()); + h.extend_from_slice(&0u64.to_le_bytes()); // tensor count + h.extend_from_slice(&3u64.to_le_bytes()); // kv count + gguf_str(&mut h, "tokenizer.ggml.tokens"); + h.extend_from_slice(&9u32.to_le_bytes()); // array + h.extend_from_slice(&8u32.to_le_bytes()); // of strings + h.extend_from_slice(&3u64.to_le_bytes()); + for t in ["<|im_start|>", "hello", "world"] { + gguf_str(&mut h, t); + } + gguf_str(&mut h, "qwen3moe.expert_count"); + h.extend_from_slice(&4u32.to_le_bytes()); // u32 + h.extend_from_slice(&128u32.to_le_bytes()); + gguf_str(&mut h, "general.architecture"); + h.extend_from_slice(&8u32.to_le_bytes()); // string + gguf_str(&mut h, arch); + h + } + + fn write(tag: &str, bytes: &[u8]) -> std::path::PathBuf { + let p = std::env::temp_dir().join(format!("moe-pred-{tag}-{}", std::process::id())); + std::fs::write(&p, bytes).expect("write fixture"); + p + } + + /// #3714: routed-expert files are recognised from the header alone; dense + /// ones, non-GGUF files and truncated headers are not. + #[test] + fn the_moe_predicate_reads_only_the_header() { + let moe = write("moe", &header_with_arch_last("qwen3moe")); + let dense = write("dense", &header_with_arch_last("qwen3")); + let bad = write("bad", b"NOTGGUF-at-all"); + let mut cut = header_with_arch_last("qwen3moe"); + cut.truncate(cut.len() - 3); + let cut = write("cut", &cut); + assert!( + moe_loader_architecture(&moe), + "qwen3moe is the routed-expert arch" + ); + assert!(!moe_loader_architecture(&dense), "dense qwen3 is not"); + assert!(!moe_loader_architecture(&bad), "a non-GGUF file is not"); + assert!( + !moe_loader_architecture(&cut), + "a truncated header is not, never a guess" + ); + for p in [moe, dense, bad, cut] { + let _ = std::fs::remove_file(p); + } + } + + /// The real file, when `APR_QWEN3MOE_GGUF` names one. Run under + /// `/usr/bin/time -v` for the peak-RSS row the #3714 receipt cites. + #[test] + fn the_moe_predicate_answers_the_real_file_from_its_header() { + let Some(path) = std::env::var_os("APR_QWEN3MOE_GGUF") else { + eprintln!("SKIP: APR_QWEN3MOE_GGUF is not set"); + return; + }; + assert!(moe_loader_architecture(std::path::Path::new(&path))); + } +} diff --git a/crates/apr-cli/src/commands/qa_failed_gates_gate.rs b/crates/apr-cli/src/commands/qa_failed_gates_gate.rs index ec95354bd1..a664570114 100644 --- a/crates/apr-cli/src/commands/qa_failed_gates_gate.rs +++ b/crates/apr-cli/src/commands/qa_failed_gates_gate.rs @@ -436,3 +436,31 @@ assert!((config.min_speedup - 0.2).abs() < f64::EPSILON); assert!((config.min_gpu_speedup - 2.0).abs() < f64::EPSILON); } + + /// #3714 done_when 3: a gate whose runner ERRORS becomes a FAILED row and + /// the gates after it still run — `apr qa` never abandons its report. The + /// pre-fix `runner()?` returned the error out of `run_qa`, and on a + /// qwen3moe file `apr qa --json` printed zero bytes. + #[test] + fn dispatch_gate_records_an_erroring_gate_as_failed_and_continues() { + let mut gates = Vec::new(); + dispatch_gate(&mut gates, true, false, "golden_output", "", || { + Err(CliError::ValidationFailed("CPU generation failed: EMPTY data buffer".into())) + }) + .expect("an erroring gate is a row, not an abort"); + dispatch_gate(&mut gates, true, false, "throughput", "", || { + Ok(GateResult::passed("throughput", "ok", None, None, Duration::from_secs(1))) + }) + .expect("the next gate runs"); + assert_eq!(gates.len(), 2, "both gates are in the report"); + let golden = &gates[0]; + assert_eq!(golden.name, "golden_output"); + assert!(!golden.passed, "an error is a FAIL"); + assert!(!golden.skipped, "an error is never a skip"); + assert!( + golden.message.contains("EMPTY data buffer"), + "the row carries the error text: {}", + golden.message + ); + assert!(gates[1].passed); + } diff --git a/crates/apr-cli/src/commands/qa_gguf.rs b/crates/apr-cli/src/commands/qa_gguf.rs index 3e8e80778a..803c160b7b 100644 --- a/crates/apr-cli/src/commands/qa_gguf.rs +++ b/crates/apr-cli/src/commands/qa_gguf.rs @@ -3,13 +3,10 @@ fn is_gguf_format(path: &Path) -> bool { #[cfg(feature = "inference")] { use realizar::format::{detect_format, ModelFormat}; - let magic = std::fs::read(path).ok().and_then(|b| { - if b.len() >= 8 { - Some(b[..8].to_vec()) - } else { - None - } - }); + // #3750: the 8-byte magic, never the whole model + let magic = super::model_header::read_prefix(path, 8) + .ok() + .filter(|b| b.len() >= 8); magic.and_then(|m| detect_format(&m).ok()) == Some(ModelFormat::Gguf) } #[cfg(not(feature = "inference"))] @@ -78,6 +75,27 @@ const HYBRID_DENSE_GPU_SKIP: &str = "Gated DeltaNet runs on the GPU (#3090), but this gate measures through the dense \ OwnedQuantizedModelCuda, which has no hybrid path"; +/// Skip reason for a gate measured through the dense `OwnedQuantizedModel` / +/// `OwnedQuantizedModelCuda` on a Qwen3-MoE file (#3714). +/// +/// Not "the GPU declined": the routed-expert forward runs on CUDA (#3714) and +/// on the CPU (#3367), and the Golden Output gate judges it through the runtime +/// entry point. What this gate lacks is its own instrument: the dense loader +/// fills the MoE layers' FFN with a placeholder its forward cannot read. +const MOE_DENSE_SKIP: &str = + "Dense-loader gate: qwen3moe runs through the routed-expert forward (CUDA #3714, CPU \ + #3367), but this gate measures through the dense OwnedQuantizedModel, whose FFN is the \ + MoE placeholder"; + +/// Skip decision for a dense-instrument gate on a Qwen3-MoE file. +fn moe_gate_skip(moe: bool, skip: bool, reason: &'static str) -> (bool, &'static str) { + if moe { + (true, MOE_DENSE_SKIP) + } else { + (skip, reason) + } +} + /// Skip decision for a GPU gate on a hybrid file. fn hybrid_gpu_gate_skip(hybrid: bool, skip: bool, reason: &'static str) -> (bool, &'static str) { if hybrid { @@ -206,6 +224,10 @@ fn run_qa(path: &Path, config: &QaConfig) -> Result { // #3477: and independently of which backend runs it — the dense loader has // no hybrid path, so every gate built on it skips with that reason. let hybrid = super::qa_capability::hybrid_loader_architecture(path); + // #3714: a Qwen3-MoE file loads through the dense loader but only the + // routed-expert forward can run it — the golden and throughput gates go + // through the runtime entry point, and dense-instrument gates skip. + let moe = super::qa_capability::moe_loader_architecture(path); let get_skip = |skip: bool, reason: &'static str| -> (bool, &'static str) { if capability_match_failed { @@ -219,12 +241,12 @@ fn run_qa(path: &Path, config: &QaConfig) -> Result { // entry point `apr run` uses, because the dense loader refuses the model. let (s, r) = get_skip(config.skip_golden, "Skipped by --skip-golden"); dispatch_gate(&mut gates, config.json, s, "golden_output", r, || { - golden_gate_for(cpu_only, hybrid, path, config) + golden_gate_for(cpu_only, hybrid || moe, path, config) })?; let (s, r) = get_skip(config.skip_throughput, "Skipped by --skip-throughput"); dispatch_gate(&mut gates, config.json, s, "throughput", r, || { - throughput_gate_for(cpu_only, hybrid, path, config) + throughput_gate_for(cpu_only, hybrid || moe, path, config) })?; let is_ollama_fmt = is_gguf_format(path); @@ -236,6 +258,7 @@ fn run_qa(path: &Path, config: &QaConfig) -> Result { }; let (s, r) = get_skip(orig_skip_ollama, orig_reason_ollama); let (s, r) = dense_gate_skip(cpu_only || hybrid, s, r); + let (s, r) = moe_gate_skip(moe, s, r); dispatch_gate(&mut gates, config.json, s, "ollama_parity", r, || { run_ollama_parity_gate(path, config) })?; @@ -243,13 +266,18 @@ fn run_qa(path: &Path, config: &QaConfig) -> Result { let (s, r) = get_skip(config.skip_gpu_speedup, "Skipped by --skip-gpu-speedup"); let (s, r) = gpu_gate_skip(cpu_only, s, r); let (s, r) = hybrid_gpu_gate_skip(hybrid, s, r); + let (s, r) = moe_gate_skip(moe, s, r); dispatch_gate(&mut gates, config.json, s, "gpu_speedup", r, || { run_gpu_speedup_gate(path, config) })?; // get_skip expects &'static str, but the format-parity reason is &str, so // the three-way decision lives in its own function. - let (s, r) = format_parity_skip(capability_match_failed, cpu_only || hybrid, config); + let (s, r) = if moe && !capability_match_failed { + (true, MOE_DENSE_SKIP) + } else { + format_parity_skip(capability_match_failed, cpu_only || hybrid, config) + }; dispatch_gate(&mut gates, config.json, s, "format_parity", r, || { run_format_parity_gate(path, config) })?; @@ -265,6 +293,7 @@ fn run_qa(path: &Path, config: &QaConfig) -> Result { let (s, r) = get_skip(config.skip_gpu_state, "Skipped by --skip-gpu-state"); let (s, r) = gpu_gate_skip(cpu_only, s, r); let (s, r) = hybrid_gpu_gate_skip(hybrid, s, r); + let (s, r) = moe_gate_skip(moe, s, r); dispatch_gate(&mut gates, config.json, s, "gpu_state_isolation", r, || { run_gpu_state_isolation_gate(path, config) })?; diff --git a/crates/apr-cli/src/commands/qa_tests_report_pass_fail.rs b/crates/apr-cli/src/commands/qa_tests_report_pass_fail.rs index c407179fe5..098337366c 100644 --- a/crates/apr-cli/src/commands/qa_tests_report_pass_fail.rs +++ b/crates/apr-cli/src/commands/qa_tests_report_pass_fail.rs @@ -603,3 +603,27 @@ "Empty temp file should map to 0.5b via file size heuristic" ); } + + /// #3714: the throughput label names what the timed runs REPORTED. A cuda + /// build whose runs all fell back to the CPU must not print "GPU". + #[cfg(feature = "inference")] + #[test] + fn runtime_backend_label_follows_used_gpu_not_the_build() { + assert_eq!( + runtime_backend_label(3, 3), + "runtime entry point, GPU on every timed run" + ); + assert_eq!( + runtime_backend_label(0, 3), + "runtime entry point, CPU on every timed run" + ); + assert_eq!( + runtime_backend_label(1, 3), + "runtime entry point, MIXED GPU/CPU timed runs" + ); + assert_eq!(runtime_backend_label(0, 0), "runtime entry point, no timed run"); + for (g, r) in [(0, 1), (0, 3), (1, 3), (2, 3)] { + assert!(!runtime_backend_label(g, r).contains("GPU on every"), "{g}/{r}"); + } + } + diff --git a/crates/apr-cli/src/commands/rosetta.rs b/crates/apr-cli/src/commands/rosetta.rs index b8797b2934..4c4a46cb9a 100644 --- a/crates/apr-cli/src/commands/rosetta.rs +++ b/crates/apr-cli/src/commands/rosetta.rs @@ -13,6 +13,9 @@ use aprender::format::rosetta::{ ConversionOptions, ConversionPath, ConversionReport, FormatType, InspectionReport, RosettaStone, TensorInfo, VerificationReport, }; +use batuta_common::cli_roles::{ + ConfigPath, DirPath, FreeText, InputFile, ModelPath, OutputPath, PromptText, +}; use clap::Subcommand; use colored::Colorize; use std::fmt::Write; @@ -25,7 +28,7 @@ pub enum RosettaCommands { Inspect { /// Path to model file (GGUF, SafeTensors, or APR) #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Show hex dump of file header #[arg(long)] @@ -40,15 +43,15 @@ pub enum RosettaCommands { Convert { /// Source model file #[arg(value_name = "SOURCE")] - source: PathBuf, + source: ModelPath, /// Target output file (format inferred from extension) #[arg(value_name = "TARGET")] - target: PathBuf, + target: OutputPath, /// Apply quantization during conversion (int8, int4, fp16) #[arg(long)] - quantize: Option, + quantize: Option, /// Verify conversion with round-trip check #[arg(long)] @@ -60,22 +63,22 @@ pub enum RosettaCommands { /// External tokenizer.json for weights-only models (PMAT-232) #[arg(long, value_name = "TOKENIZER")] - tokenizer: Option, + tokenizer: Option, }, /// Execute multi-step conversion chain Chain { /// Source model file #[arg(value_name = "SOURCE")] - source: PathBuf, + source: ModelPath, /// Format chain (e.g., gguf safetensors apr) #[arg(value_name = "FORMATS", num_args = 2..)] - formats: Vec, + formats: Vec, /// Working directory for intermediate files #[arg(long, default_value = "./rosetta-work")] - work_dir: PathBuf, + work_dir: DirPath, /// Output as JSON #[arg(long)] @@ -86,11 +89,11 @@ pub enum RosettaCommands { Verify { /// Source model file #[arg(value_name = "SOURCE")] - source: PathBuf, + source: ModelPath, /// Intermediate format for round-trip (gguf, safetensors, apr) #[arg(long, default_value = "safetensors")] - intermediate: String, + intermediate: FreeText, /// Tolerance for numerical differences (default: 1e-5) #[arg(long, default_value = "1e-5", @@ -103,25 +106,26 @@ pub enum RosettaCommands { }, /// Compare inference outputs between two models (PMAT-114) + #[command(groups(batuta_common::cli_roles::SamplingArg::groups()))] CompareInference { /// Reference model (typically GGUF) #[arg(value_name = "MODEL_A")] - model_a: PathBuf, + model_a: ModelPath, /// Test model (typically APR) #[arg(value_name = "MODEL_B")] - model_b: PathBuf, + model_b: ModelPath, /// Test prompt #[arg(long, default_value = "2+2=")] - prompt: String, + prompt: PromptText, /// Maximum tokens to generate #[arg(long, default_value = "5")] max_tokens: usize, /// Sampling temperature (0 = greedy) - #[arg(long, default_value = "0")] + #[arg(long, default_value = "0", group = batuta_common::cli_roles::SamplingKind::Temperature.id())] temperature: f32, /// Logit difference tolerance @@ -142,11 +146,11 @@ pub enum RosettaCommands { DiffTensors { /// Reference model (typically GGUF - the one that works) #[arg(value_name = "MODEL_A")] - model_a: PathBuf, + model_a: ModelPath, /// Test model (typically APR - the one producing garbage) #[arg(value_name = "MODEL_B")] - model_b: PathBuf, + model_b: ModelPath, /// Only show tensors with dimension mismatches #[arg(long)] @@ -158,7 +162,7 @@ pub enum RosettaCommands { /// Filter tensors by name pattern (e.g., "embed", "lm_head", "layer.0") #[arg(long)] - filter: Option, + filter: Option, /// Output as JSON #[arg(long)] @@ -173,19 +177,19 @@ pub enum RosettaCommands { Fingerprint { /// Model file to fingerprint #[arg(value_name = "MODEL")] - model: PathBuf, + model: ModelPath, /// Second model to compare (optional - enables diff mode) #[arg(value_name = "MODEL_B")] - model_b: Option, + model_b: Option, /// Output fingerprints to JSON file #[arg(long, short)] - output: Option, + output: Option, /// Filter tensors by name pattern #[arg(long)] - filter: Option, + filter: Option, /// Show detailed statistics for each tensor #[arg(long)] @@ -203,15 +207,15 @@ pub enum RosettaCommands { ValidateStats { /// Model to validate #[arg(value_name = "MODEL")] - model: PathBuf, + model: ModelPath, /// Reference model for comparison #[arg(long)] - reference: Option, + reference: Option, /// Fingerprint JSON file for comparison #[arg(long)] - fingerprints: Option, + fingerprints: Option, /// Deviation threshold in standard deviations (default: 3.0) #[arg(long, default_value = "3.0", diff --git a/crates/apr-cli/src/commands/rosetta_fingerprints_load.rs b/crates/apr-cli/src/commands/rosetta_fingerprints_load.rs index 4891db0b1e..bc195aa79d 100644 --- a/crates/apr-cli/src/commands/rosetta_fingerprints_load.rs +++ b/crates/apr-cli/src/commands/rosetta_fingerprints_load.rs @@ -53,9 +53,9 @@ #[test] fn test_rosetta_commands_compare_inference() { let cmd = RosettaCommands::CompareInference { - model_a: PathBuf::from("model_a.gguf"), - model_b: PathBuf::from("model_b.apr"), - prompt: "Hello".to_string(), + model_a: PathBuf::from("model_a.gguf").into(), + model_b: PathBuf::from("model_b.apr").into(), + prompt: "Hello".to_string().into(), max_tokens: 10, temperature: 0.0, tolerance: 0.1, @@ -77,11 +77,11 @@ #[test] fn test_rosetta_commands_diff_tensors() { let cmd = RosettaCommands::DiffTensors { - model_a: PathBuf::from("model_a.gguf"), - model_b: PathBuf::from("model_b.apr"), + model_a: PathBuf::from("model_a.gguf").into(), + model_b: PathBuf::from("model_b.apr").into(), mismatches_only: true, show_values: 5, - filter: Some("attention".to_string()), + filter: Some("attention".to_string().into()), json: false, }; match cmd { @@ -93,7 +93,7 @@ } => { assert!(mismatches_only); assert_eq!(show_values, 5); - assert_eq!(filter, Some("attention".to_string())); + assert_eq!(filter, Some("attention".to_string()).map(Into::into)); } _ => panic!("Wrong command variant"), } @@ -102,9 +102,9 @@ #[test] fn test_rosetta_commands_fingerprint() { let cmd = RosettaCommands::Fingerprint { - model: PathBuf::from("model.gguf"), + model: PathBuf::from("model.gguf").into(), model_b: None, - output: Some(PathBuf::from("fingerprints.json")), + output: Some(PathBuf::from("fingerprints.json").into()), filter: None, verbose: true, json: false, @@ -123,9 +123,9 @@ #[test] fn test_rosetta_commands_validate_stats() { let cmd = RosettaCommands::ValidateStats { - model: PathBuf::from("model.gguf"), + model: PathBuf::from("model.gguf").into(), reference: None, - fingerprints: Some(PathBuf::from("ref.json")), + fingerprints: Some(PathBuf::from("ref.json").into()), threshold: 0.01, strict: true, json: true, diff --git a/crates/apr-cli/src/commands/rosetta_rosetta.rs b/crates/apr-cli/src/commands/rosetta_rosetta.rs index aae267b131..f6107d9718 100644 --- a/crates/apr-cli/src/commands/rosetta_rosetta.rs +++ b/crates/apr-cli/src/commands/rosetta_rosetta.rs @@ -7,7 +7,7 @@ fn test_rosetta_commands_inspect_default() { // Test that the Inspect variant can be created let cmd = RosettaCommands::Inspect { - file: PathBuf::from("model.gguf"), + file: PathBuf::from("model.gguf").into(), hexdump: false, json: false, }; @@ -28,8 +28,8 @@ #[test] fn test_rosetta_commands_convert() { let cmd = RosettaCommands::Convert { - source: PathBuf::from("model.gguf"), - target: PathBuf::from("model.apr"), + source: PathBuf::from("model.gguf").into(), + target: PathBuf::from("model.apr").into(), quantize: None, verify: false, json: false, @@ -53,9 +53,9 @@ #[test] fn test_rosetta_commands_chain() { let cmd = RosettaCommands::Chain { - source: PathBuf::from("model.gguf"), - formats: vec!["safetensors".to_string(), "apr".to_string()], - work_dir: PathBuf::from("./work"), + source: PathBuf::from("model.gguf").into(), + formats: vec!["safetensors".to_string().into(), "apr".to_string().into()], + work_dir: PathBuf::from("./work").into(), json: false, }; match cmd { @@ -71,8 +71,8 @@ #[test] fn test_rosetta_commands_verify() { let cmd = RosettaCommands::Verify { - source: PathBuf::from("model.gguf"), - intermediate: "safetensors".to_string(), + source: PathBuf::from("model.gguf").into(), + intermediate: "safetensors".to_string().into(), tolerance: 1e-5, json: false, }; diff --git a/crates/apr-cli/src/commands/rosetta_tensor_fingerprint_statistical.rs b/crates/apr-cli/src/commands/rosetta_tensor_fingerprint_statistical.rs index f19225200f..0f498c1fbe 100644 --- a/crates/apr-cli/src/commands/rosetta_tensor_fingerprint_statistical.rs +++ b/crates/apr-cli/src/commands/rosetta_tensor_fingerprint_statistical.rs @@ -215,7 +215,7 @@ #[test] fn test_rosetta_commands_inspect_with_hexdump() { let cmd = RosettaCommands::Inspect { - file: PathBuf::from("model.gguf"), + file: PathBuf::from("model.gguf").into(), hexdump: true, json: true, }; @@ -231,9 +231,9 @@ #[test] fn test_rosetta_commands_convert_with_all_options() { let cmd = RosettaCommands::Convert { - source: PathBuf::from("in.safetensors"), - target: PathBuf::from("out.apr"), - quantize: Some("int4".to_string()), + source: PathBuf::from("in.safetensors").into(), + target: PathBuf::from("out.apr").into(), + quantize: Some("int4".to_string().into()), verify: true, json: true, tokenizer: None, @@ -245,7 +245,7 @@ json, .. } => { - assert_eq!(quantize, Some("int4".to_string())); + assert_eq!(quantize, Some("int4".to_string()).map(Into::into)); assert!(verify); assert!(json); } @@ -256,10 +256,10 @@ #[test] fn test_rosetta_commands_fingerprint_with_model_b() { let cmd = RosettaCommands::Fingerprint { - model: PathBuf::from("model_a.gguf"), - model_b: Some(PathBuf::from("model_b.apr")), + model: PathBuf::from("model_a.gguf").into(), + model_b: Some(PathBuf::from("model_b.apr").into()), output: None, - filter: Some("attn".to_string()), + filter: Some("attn".to_string().into()), verbose: false, json: true, }; @@ -271,7 +271,7 @@ .. } => { assert!(model_b.is_some()); - assert_eq!(filter, Some("attn".to_string())); + assert_eq!(filter, Some("attn".to_string()).map(Into::into)); assert!(json); } _ => panic!("Wrong command variant"), diff --git a/crates/apr-cli/src/commands/serve/handlers.rs b/crates/apr-cli/src/commands/serve/handlers.rs index 5e696eaeef..4e2f2382bd 100644 --- a/crates/apr-cli/src/commands/serve/handlers.rs +++ b/crates/apr-cli/src/commands/serve/handlers.rs @@ -781,6 +781,8 @@ fn serve_wgpu_backend( dims.head_dim, dims.intermediate_dim, ); + // #4056: the WGSL RMSNorm takes the model's eps (it hardcoded 1e-6). + fwd.set_rms_norm_eps(quantized.config().eps); upload_wgpu_weights(&mut fwd, quantized, weights, num_layers); diff --git a/crates/apr-cli/src/commands/serve/mod.rs b/crates/apr-cli/src/commands/serve/mod.rs index 0679436a4a..3ff46d9828 100644 --- a/crates/apr-cli/src/commands/serve/mod.rs +++ b/crates/apr-cli/src/commands/serve/mod.rs @@ -20,6 +20,11 @@ pub use types::*; // Test modules #[cfg(test)] mod tests; +// #3809: the apr-serve-v1 / http-api-v1 FALSIFY-SRV/HTTP enforcement suite. It sat +// beside `tests.rs` unwired, so its 25 tests never compiled and the contract's ids +// pointed at code that never ran. +#[cfg(test)] +mod tests_contract_enforcement; // PP-LLAMA-001 PP-14/PP-15/§9 #8: the offload report the served process // publishes. `inference`-gated because the report type comes from realizar. #[cfg(all(test, feature = "inference"))] @@ -206,8 +211,8 @@ pub(crate) fn resolve_gpu_layers( /// /// `cargo install aprender` produces exactly that build — root `Cargo.toml` has /// `default = ["cli"]` and `cuda` is opt-in. Measured on 2026-08-24 with an -/// idle RTX 4090 in the machine: 15.7 tok/s decode against llama.cpp's 158.9, -/// and 7.5 SECONDS to first token. A tenth of the speed, no diagnostic, and a +/// idle RTX 4090 in the machine (the numbers are on #2696): about a tenth of +/// llama.cpp's decode rate, and 7.5 SECONDS to first token. A tenth of the speed, no diagnostic, and a /// plausible-looking number at the end of it. /// /// The remedy in the message is checked to be real. #2527 is the counter-case: diff --git a/crates/apr-cli/src/commands/serve/tests_contract_enforcement.rs b/crates/apr-cli/src/commands/serve/tests_contract_enforcement.rs index c3dd427a6b..704836f922 100644 --- a/crates/apr-cli/src/commands/serve/tests_contract_enforcement.rs +++ b/crates/apr-cli/src/commands/serve/tests_contract_enforcement.rs @@ -13,14 +13,16 @@ use std::sync::Arc; fn create_test_model_file() -> tempfile::NamedTempFile { let mut file = tempfile::NamedTempFile::new().expect("tempfile::NamedTempFile::new()"); - file.write_all(b"test model data").expect("write_all(b'test model data'"); + file.write_all(b"test model data") + .expect("write_all(b'test model data'"); file } fn create_test_state() -> Arc { let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); // Keep the temp file alive by leaking the path std::mem::forget(model); Arc::new(state) @@ -34,7 +36,8 @@ fn falsify_srv_001_health_returns_503_when_not_ready() { // ServerState starts with ready=false. health_check() must return Unhealthy. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); // State starts not ready assert!( !state.is_ready(), @@ -53,7 +56,8 @@ fn falsify_srv_001b_health_returns_200_when_ready() { // FALSIFY-SRV-001b: /health returns Healthy once ready=true. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); state.ready.store(true, Ordering::Release); assert!(state.is_ready()); let health = health_check(&state); @@ -182,7 +186,8 @@ fn falsify_srv_006_health_response_fields() { // FALSIFY-SRV-006: HealthResponse has all required fields per contract. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); state.ready.store(true, Ordering::Release); let health = health_check(&state); // All fields must be present and valid @@ -205,14 +210,17 @@ fn falsify_srv_006_health_response_fields() { fn falsify_srv_007_health_status_serde() { // FALSIFY-SRV-007: HealthStatus enum serializes to lowercase strings. // Contract: "healthy", "degraded", "unhealthy" — not "Healthy", etc. - let json = serde_json::to_string(&HealthStatus::Healthy).expect("serde_json::to_string(&HealthS"); + let json = + serde_json::to_string(&HealthStatus::Healthy).expect("serde_json::to_string(&HealthS"); assert_eq!(json, "\"healthy\"", "FALSIFY-SRV-007: Healthy → lowercase"); - let json = serde_json::to_string(&HealthStatus::Degraded).expect("serde_json::to_string(&HealthS"); + let json = + serde_json::to_string(&HealthStatus::Degraded).expect("serde_json::to_string(&HealthS"); assert_eq!( json, "\"degraded\"", "FALSIFY-SRV-007: Degraded → lowercase" ); - let json = serde_json::to_string(&HealthStatus::Unhealthy).expect("serde_json::to_string(&HealthS"); + let json = + serde_json::to_string(&HealthStatus::Unhealthy).expect("serde_json::to_string(&HealthS"); assert_eq!( json, "\"unhealthy\"", "FALSIFY-SRV-007: Unhealthy → lowercase" @@ -380,7 +388,8 @@ fn falsify_srv_010_graceful_shutdown_ready_flag() { // When shutdown is initiated, health endpoint must return 503. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); // Simulate Ready → Draining transition state.ready.store(true, Ordering::Release); @@ -405,7 +414,8 @@ fn falsify_srv_011_degraded_health_on_high_latency() { // Contract: server_lifecycle invariant — Degraded when p99 > 1s. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); state.ready.store(true, Ordering::Release); // Record 10 requests with 2000ms average latency @@ -462,7 +472,8 @@ fn falsify_srv_013_mmap_threshold() { // Contract: ServerState determines mmap based on 50MB threshold. let model = create_test_model_file(); let config = ServerConfig::default(); - let state = ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); + let state = + ServerState::new(model.path().to_path_buf(), config).expect("to_path_buf(), config"); // Test file is small — should NOT use mmap assert!( diff --git a/crates/apr-cli/src/commands/speedup.rs b/crates/apr-cli/src/commands/speedup.rs index c5bd5e1173..1ffc1a3dd8 100644 --- a/crates/apr-cli/src/commands/speedup.rs +++ b/crates/apr-cli/src/commands/speedup.rs @@ -46,16 +46,16 @@ fn run_throughput_gate(path: &Path, config: &QaConfig) -> Result { #[cfg(not(feature = "cuda"))] let cuda_available = false; - let model_bytes = std::fs::read(path) + // #3750: the format from the 8-byte magic, never the whole model + let magic = super::model_header::read_prefix(path, 8) .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let format = detect_format(&model_bytes[..8.min(model_bytes.len())]) + let format = detect_format(&magic) .map_err(|e| CliError::ValidationFailed(format!("Failed to detect format: {e}")))?; let prompt = "Write a hello world program in Python:"; let Some((tps, _measurement_duration)) = throughput_for_format( path, - &model_bytes, format, prompt, config, @@ -119,7 +119,7 @@ fn run_throughput_gate(path: &Path, config: &QaConfig) -> Result { /// The threshold is untouched (10 tok/s for unasserted GGUF); what changes is /// that the number the gate compares is decode throughput. #[cfg(feature = "inference")] -fn throughput_runtime(path: &Path, config: &QaConfig) -> Result { +fn throughput_runtime(path: &Path, config: &QaConfig) -> Result<(f64, &'static str)> { use realizar::{run_inference, InferenceConfig}; let infer_config = InferenceConfig::new(path) @@ -139,17 +139,34 @@ fn throughput_runtime(path: &Path, config: &QaConfig) -> Result { let mut generated = 0usize; let mut seconds = 0.0_f64; + let (mut gpu_runs, mut runs) = (0usize, 0usize); for _ in 0..config.iterations.max(1) { let result = run()?; generated += result.generated_token_count; seconds += result.inference_ms / 1000.0; + runs += 1; + gpu_runs += usize::from(result.used_gpu); } - Ok(if seconds > 0.0 { + let tps = if seconds > 0.0 { generated as f64 / seconds } else { 0.0 - }) + }; + Ok((tps, runtime_backend_label(gpu_runs, runs))) +} + +/// #3714: the backend the timed runs REPORTED (`used_gpu`), never the build's +/// features — a cuda build whose device could not serve ran on the CPU, and the +/// gate used to print "GPU" for it (and "hybrid" for a MoE file). +#[cfg(feature = "inference")] +fn runtime_backend_label(gpu_runs: usize, runs: usize) -> &'static str { + match (gpu_runs, runs) { + (_, 0) => "runtime entry point, no timed run", + (g, r) if g == r => "runtime entry point, GPU on every timed run", + (0, _) => "runtime entry point, CPU on every timed run", + _ => "runtime entry point, MIXED GPU/CPU timed runs", + } } /// Gate 2 for an architecture the dense loader refuses: the same falsifiable @@ -168,14 +185,9 @@ fn run_throughput_gate_runtime(path: &Path, config: &QaConfig) -> Result= threshold { ">=" } else { "<" } @@ -245,10 +257,10 @@ fn measure_our_gguf_tps(path: &Path, config: &QaConfig, tracer: &TracerImpl) -> GGUFModel, MappedGGUFModel, OwnedQuantizedModel, QuantizedGenerateConfig, }; - let model_bytes = std::fs::read(path) - .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let gguf = GGUFModel::from_bytes(&model_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; + // #3750: one map; the tokenizer comes from its header, not from a whole-file read + let mapped = MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; + let gguf = &mapped.model; let prompt = "Write a function to check if a number is prime:"; let bos = aprender::demo::SpecialTokens::qwen2().bos_id; @@ -270,8 +282,6 @@ fn measure_our_gguf_tps(path: &Path, config: &QaConfig, tracer: &TracerImpl) -> #[cfg(not(feature = "cuda"))] let cuda_available = false; - let mapped = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; let model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("Model failed: {e}")))?; @@ -416,10 +426,10 @@ fn measure_gpu_cpu_tps(path: &Path, config: &QaConfig, tracer: &TracerImpl) -> R QuantizedGenerateConfig, }; - let model_bytes = std::fs::read(path) - .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let gguf = GGUFModel::from_bytes(&model_bytes) - .map_err(|e| CliError::ValidationFailed(format!("Failed to parse GGUF: {e}")))?; + // #3750: one map; the tokenizer comes from its header, not from a whole-file read + let mapped = MappedGGUFModel::from_path(path) + .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; + let gguf = &mapped.model; let prompt = "Write a function to calculate factorial:"; let bos = aprender::demo::SpecialTokens::qwen2().bos_id; @@ -433,8 +443,6 @@ fn measure_gpu_cpu_tps(path: &Path, config: &QaConfig, tracer: &TracerImpl) -> R let budget_us = config.max_tokens as u64 * config.iterations as u64 * 100_000; // CPU throughput - let mapped = MappedGGUFModel::from_path(path) - .map_err(|e| CliError::ValidationFailed(format!("Map failed: {e}")))?; let model = OwnedQuantizedModel::from_mapped(&mapped) .map_err(|e| CliError::ValidationFailed(format!("Model failed: {e}")))?; let (cpu_tps, _) = measure_generate_throughput( @@ -501,9 +509,10 @@ fn run_gpu_speedup_gate(path: &Path, config: &QaConfig) -> Result { )); } - let model_bytes = std::fs::read(path) + // #3750: the format from the 8-byte magic, never the whole model + let magic = super::model_header::read_prefix(path, 8) .map_err(|e| CliError::ValidationFailed(format!("Failed to read model: {e}")))?; - let format = detect_format(&model_bytes[..8.min(model_bytes.len())]) + let format = detect_format(&magic) .map_err(|e| CliError::ValidationFailed(format!("Failed to detect format: {e}")))?; if format != ModelFormat::Gguf { return Ok(GateResult::skipped( diff --git a/crates/apr-cli/src/commands/test_llm.rs b/crates/apr-cli/src/commands/test_llm.rs index fb0cc47d36..35e1f21275 100644 --- a/crates/apr-cli/src/commands/test_llm.rs +++ b/crates/apr-cli/src/commands/test_llm.rs @@ -22,6 +22,9 @@ //! `decode_tok_per_sec` = 1000 / `itl_p50_ms`, the inter-token rate, which //! excludes time-to-first-token by construction and so cannot absorb a model //! load. Reading a field beats deriving a number. +// #4041: a `--no-default-features` build has no tokio, so `dispatch` cannot run a benchmark and the benchmark +// code is unreachable there. Dead in that build only; every build with `inference` still lints it. +#![cfg_attr(not(feature = "inference"), allow(dead_code))] use crate::error::{CliError, Result}; use apr_test::llm::{ benchmark::{Benchmark, BenchmarkConfig, BenchmarkReport}, @@ -411,6 +414,7 @@ use crate::LlmSubcommand; /// /// # Errors /// Propagates whichever mode ran. +#[cfg(feature = "inference")] pub fn dispatch(command: &LlmSubcommand) -> Result<()> { let rt = tokio::runtime::Runtime::new() .map_err(|e| CliError::InferenceFailed(format!("tokio runtime: {e}")))?; @@ -420,6 +424,18 @@ pub fn dispatch(command: &LlmSubcommand) -> Result<()> { } } +/// #4041: the benchmark drives its server through tokio, which only the `inference` feature brings. A minimal +/// build (`--no-default-features`) names the missing feature instead of failing to compile. +/// +/// # Errors +/// Always: this build cannot run the benchmark. +#[cfg(not(feature = "inference"))] +pub fn dispatch(_command: &LlmSubcommand) -> Result<()> { + Err(CliError::InferenceFailed( + "`apr test llm` needs the `inference` feature (this apr was built without it)".to_string(), + )) +} + /// TWO MODES, ONE ENTRYPOINT — the §4.4-conformant one. /// /// `--band` selects PERF-024's `run_band`, which until PERF-025 was called by @@ -479,7 +495,7 @@ async fn dispatch_band(command: &LlmSubcommand) -> Result<()> { accelerator, quantization, compute_class, - server_features, + server_features: &batuta_common::cli_roles::strings(server_features), tokenization, tokenizer_sha256: tokenizer_sha256.as_deref(), counts_special_tokens: *counts_special_tokens, diff --git a/crates/apr-cli/src/commands/test_llm_band.rs b/crates/apr-cli/src/commands/test_llm_band.rs index aecf2406a1..5d1ad0fde5 100644 --- a/crates/apr-cli/src/commands/test_llm_band.rs +++ b/crates/apr-cli/src/commands/test_llm_band.rs @@ -51,6 +51,9 @@ //! `receipt.r1.json` … `receipt.rN.json`. The cell's verdict is the conjunction //! over them, which is a decision for the runner, not something this producer //! should pre-collapse. +// #4041: a `--no-default-features` build has no tokio, so `dispatch` cannot run a benchmark and the benchmark +// code is unreachable there. Dead in that build only; every build with `inference` still lints it. +#![cfg_attr(not(feature = "inference"), allow(dead_code))] use crate::error::{CliError, Result}; use apr_test::llm::band::{run_band, BandRun, RequestExtra}; @@ -1770,7 +1773,12 @@ async fn cool(first_lane: &mut bool, cooldown: std::time::Duration, interleaved: *first_lane = false; return; } + #[cfg(feature = "inference")] tokio::time::sleep(cooldown).await; + // #4041: no tokio in a minimal build. Nothing reaches this there (`dispatch` refuses first); if it did, the + // cooldown would still be honoured, blocking. + #[cfg(not(feature = "inference"))] + std::thread::sleep(cooldown); } /// Run the §5.1 protocol over every requested band and write the receipts. diff --git a/crates/apr-cli/src/commands/throughput.rs b/crates/apr-cli/src/commands/throughput.rs index f3e46bb6ff..4d39e1e013 100644 --- a/crates/apr-cli/src/commands/throughput.rs +++ b/crates/apr-cli/src/commands/throughput.rs @@ -51,7 +51,6 @@ fn throughput_safetensors( #[cfg(feature = "inference")] fn throughput_for_format( path: &Path, - model_bytes: &[u8], format: realizar::format::ModelFormat, prompt: &str, config: &QaConfig, @@ -62,7 +61,7 @@ fn throughput_for_format( match format { ModelFormat::Gguf => { - throughput_gguf(path, model_bytes, config, cuda_available, tracer, prompt).map(Some) + throughput_gguf(path, config, cuda_available, tracer, prompt).map(Some) } ModelFormat::Apr => throughput_apr(path, config, tracer, prompt).map(Some), ModelFormat::SafeTensors => throughput_safetensors(path, config, tracer, prompt), diff --git a/crates/apr-cli/src/commands/train_tests.rs b/crates/apr-cli/src/commands/train_tests.rs index af22b3c12c..524ce804e8 100644 --- a/crates/apr-cli/src/commands/train_tests.rs +++ b/crates/apr-cli/src/commands/train_tests.rs @@ -320,7 +320,7 @@ mod tests { }) .expect("spawn parse thread") .join() - .expect("parse thread must not panic") + .expect("parse thread must not panic").into() } #[test] diff --git a/crates/apr-cli/src/commands_enum.rs b/crates/apr-cli/src/commands_enum.rs index 8b3e7de36f..06d3f2ef13 100644 --- a/crates/apr-cli/src/commands_enum.rs +++ b/crates/apr-cli/src/commands_enum.rs @@ -102,19 +102,22 @@ pub enum CodeInputFormat { #[derive(Subcommand, Debug)] pub enum Commands { /// Run model directly (auto-download, cache, execute) + // #3745 S1 v1.1: the sampling controls below join this group, which is how + // `apr surface` reads them as role `sampling`. + #[command(groups(batuta_common::cli_roles::SamplingArg::groups()))] Run { /// Model source: local path, hf://org/repo, or URL #[arg(value_name = "SOURCE")] - source: String, + source: ModelRef, /// Text prompt (positional): `apr run model.gguf "What is 2+2?"` #[arg(value_name = "PROMPT")] - positional_prompt: Option, + positional_prompt: Option, /// Input file (audio, text, etc.) #[arg(short, long)] - input: Option, + input: Option, /// Text prompt for generation (for LLM models) #[arg(short, long)] - prompt: Option, + prompt: Option, /// Maximum tokens to generate (default: 32) #[arg(short = 'n', long, default_value = "32")] max_tokens: usize, @@ -123,10 +126,10 @@ pub enum Commands { stream: bool, /// Language code (for ASR models) #[arg(short, long)] - language: Option, + language: Option, /// Task (transcribe, translate) #[arg(short, long)] - task: Option, + task: Option, /// Output format (text, json, srt, vtt) #[arg(short = 'f', long, default_value = "text", value_parser = RUN_FORMAT_VALUES)] format: String, @@ -151,13 +154,13 @@ pub enum Commands { trace: bool, /// Trace specific steps only (comma-separated) #[arg(long, value_delimiter = ',')] - trace_steps: Option>, + trace_steps: Option>, /// Verbose tracing (show tensor values) #[arg(long)] trace_verbose: bool, /// Save trace output to JSON file #[arg(long, value_name = "FILE")] - trace_output: Option, + trace_output: Option, /// Trace detail level (none, basic, layer, payload, chrome) /// "chrome" outputs chrome://tracing JSON integrating layer trace + brick profile. /// F-CLIPARITY-01 / PMAT-386 / paiml/aprender#574 @@ -176,26 +179,26 @@ pub enum Commands { #[arg(long)] chat: bool, /// Sampling temperature (0.0 = greedy, default: 0.0) - #[arg(long, default_value = "0.0")] + #[arg(long, default_value = "0.0", group = batuta_common::cli_roles::SamplingKind::Temperature.id())] temperature: f32, /// Top-k sampling (default: 1 = greedy) - #[arg(long, default_value = "1")] + #[arg(long, default_value = "1", group = batuta_common::cli_roles::SamplingKind::TopK.id())] top_k: usize, /// Top-p nucleus sampling (0.0 = disabled). When set with --top-k, applies top-k first then top-p. /// F-CLIPARITY-01 / PMAT-381 / paiml/aprender#569 - #[arg(long)] + #[arg(long, group = batuta_common::cli_roles::SamplingKind::TopP.id())] top_p: Option, /// RNG seed for deterministic sampling (default: 299792458, matching Candle) /// F-CLIPARITY-01 / PMAT-382 / paiml/aprender#570 - #[arg(long, default_value = "299792458")] + #[arg(long, default_value = "299792458", group = batuta_common::cli_roles::SamplingKind::Seed.id())] seed: u64, /// Repetition penalty (1.0 = no penalty, >1.0 penalizes repeats) /// F-CLIPARITY-01 / PMAT-383 / paiml/aprender#571 - #[arg(long, default_value = "1.0")] + #[arg(long, default_value = "1.0", group = batuta_common::cli_roles::SamplingKind::RepeatPenalty.id())] repeat_penalty: f32, /// Context window for repetition penalty (number of recent tokens to check) /// F-CLIPARITY-01 / PMAT-384 / paiml/aprender#571 - #[arg(long, default_value = "64")] + #[arg(long, default_value = "64", group = batuta_common::cli_roles::SamplingKind::RepeatLastN.id())] repeat_last_n: usize, /// Process prompt tokens one-by-one instead of batched prefill. /// Useful for debugging prefill correctness (comparing per-token attention). @@ -207,7 +210,7 @@ pub enum Commands { /// Each input line: {"prompt": "...", "task_id": "..."} /// Chat template is applied automatically. #[arg(long, value_name = "FILE")] - batch_jsonl: Option, + batch_jsonl: Option, /// Show verbose output (model loading, backend info) #[arg(short, long)] verbose: bool, @@ -224,7 +227,7 @@ pub enum Commands { Inspect { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Show vocabulary details #[arg(long)] vocab: bool, @@ -259,7 +262,7 @@ pub enum Commands { Debug { /// Path to .apr model file (omit only when using a subcommand) #[arg(value_name = "FILE")] - file: Option, + file: Option, /// Debug subcommand, e.g. `embed-viz` #[command(subcommand)] action: Option, @@ -280,7 +283,7 @@ pub enum Commands { Validate { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Show 100-point quality assessment #[arg(long)] quality: bool, @@ -298,10 +301,10 @@ pub enum Commands { ValidateManifest { /// Path to manifest YAML #[arg(value_name = "MANIFEST")] - file: PathBuf, + file: ConfigPath, /// Optional local .apr artifact to discharge FALSIFY-PM-002 (sha256 match) #[arg(long, value_name = "APR_FILE")] - artifact: Option, + artifact: Option, /// Discharge FALSIFY-PM-003 via network: HTTP HEAD + streaming sha256. /// Default is DEFERRED (offline-safe). Ignored when --offline is set. /// Closes F-PUBLISH-EXTRA-001::dogfood_ex05 (no Python in ex-05). @@ -312,10 +315,10 @@ pub enum Commands { Diff { /// First model file #[arg(value_name = "FILE1")] - file1: PathBuf, + file1: ModelPath, /// Second model file #[arg(value_name = "FILE2")] - file2: PathBuf, + file2: ModelPath, /// Show weight-level differences #[arg(long)] weights: bool, @@ -324,7 +327,7 @@ pub enum Commands { values: bool, /// Filter tensors by name pattern (for --values) #[arg(long)] - filter: Option, + filter: Option, /// Maximum number of tensors to compare (for --values) #[arg(long, default_value = "10")] limit: usize, @@ -351,13 +354,13 @@ pub enum Commands { Tensors { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Show tensor statistics (mean, std, min, max) #[arg(long)] stats: bool, /// Filter tensors by name pattern #[arg(long)] - filter: Option, + filter: Option, /// Limit number of tensors shown (0 = unlimited) #[arg(long, default_value = "0")] limit: usize, @@ -369,13 +372,13 @@ pub enum Commands { Trace { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Filter layers by name pattern #[arg(long)] - layer: Option, + layer: Option, /// Compare with reference model #[arg(long)] - reference: Option, + reference: Option, /// Output as JSON #[arg(long)] json: bool, @@ -398,22 +401,22 @@ pub enum Commands { /// stage. Output goes to `--save-tensor-dir` if provided, /// else `/trace-tensors//`. #[arg(long, value_name = "STAGES")] - save_tensor: Option, + save_tensor: Option, /// Output directory for `--save-tensor` (default: sibling /// `trace-tensors//`). #[arg(long, value_name = "DIR")] - save_tensor_dir: Option, + save_tensor_dir: Option, /// Layer-id range for `--save-tensor` (default: 0..1, i.e. /// layer 0 only). Format: `START..END` (Rust range syntax, /// END exclusive). #[arg(long, value_name = "RANGE", default_value = "0..1")] - save_tensor_layers: String, + save_tensor_layers: FreeText, }, /// Check for best practices and conventions Lint { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Fail on warnings as well as errors. /// /// By default only ERROR-level findings fail the run. Every real model @@ -428,7 +431,7 @@ pub enum Commands { BeatRun { /// Path to a beat-benchmark contract YAML (e.g. contracts/beat-sklearn-iris-v1.yaml) #[arg(value_name = "CONTRACT")] - contract: PathBuf, + contract: ConfigPath, /// Measured metric value; when given, emit a WON/REGRESSED verdict and /// exit non-zero on regression. Omit to just report the pinned baseline. #[arg(long, value_name = "VALUE")] @@ -438,22 +441,22 @@ pub enum Commands { Manifest { /// Files to include in the manifest (one entry per file) #[arg(value_name = "FILES", num_args = 1..)] - files: Vec, + files: Vec, /// Output JSON manifest path #[arg(short, long, value_name = "MAN_JSON")] - output: PathBuf, + output: OutputPath, }, /// Explain errors, architecture, tensors, and kernel dispatch Explain { /// Error code, model file path, or family name (auto-detected) #[arg(value_name = "CODE_OR_FILE")] - code_or_file: Option, + code_or_file: Option, /// Path to .apr model file (optional context for --tensor) #[arg(short, long)] - file: Option, + file: Option, /// Explain a specific tensor #[arg(long)] - tensor: Option, + tensor: Option, /// Explain kernel dispatch pipeline for architecture #[arg(long)] kernel: bool, @@ -476,22 +479,22 @@ pub enum Commands { Export { /// Path to .apr model file #[arg(value_name = "FILE", required_unless_present = "list_formats")] - file: Option, + file: Option, /// Output format (safetensors, gguf, mlx, onnx, openvino, coreml) #[arg(long, default_value = "safetensors")] - format: String, + format: FreeText, /// Output file/directory path #[arg(short, long)] - output: Option, + output: Option, /// Apply quantization during export (int8, int4, fp16) #[arg(long)] - quantize: Option, + quantize: Option, /// List all supported export formats #[arg(long)] list_formats: bool, /// Batch export to multiple formats (comma-separated: gguf,mlx,safetensors) #[arg(long)] - batch: Option, + batch: Option, /// Output in JSON format #[arg(long)] json: bool, @@ -506,16 +509,16 @@ pub enum Commands { Import { /// Source: hf://org/repo, local file, or URL #[arg(value_name = "SOURCE")] - source: String, + source: ModelRef, /// Output .apr file path (default: derived from source name) #[arg(short, long)] - output: Option, + output: Option, /// Model architecture (whisper, llama, bert, qwen2, qwen3, gpt2, starcoder, gpt-neox, opt, phi, gemma, falcon, mamba, t5, auto) #[arg(long, default_value = "auto")] - arch: String, + arch: FreeText, /// Quantization (int8, int4, fp16) #[arg(long)] - quantize: Option, + quantize: Option, /// Strict mode: reject unverified architectures and fail on validation errors #[arg(long)] strict: bool, @@ -526,7 +529,7 @@ pub enum Commands { /// PMAT-232: External tokenizer.json for weights-only GGUF files. /// Required if the GGUF has no embedded tokenizer vocabulary. #[arg(long)] - tokenizer: Option, + tokenizer: Option, /// F-GT-001: Enforce provenance chain. Rejects pre-baked GGUF imports /// (only SafeTensors sources allowed). Ensures single-provenance testing. #[arg(long)] @@ -544,11 +547,11 @@ pub enum Commands { /// string "dataset", the next positional `repo` is the /// HuggingFace dataset repo and dataset-pull semantics apply. #[arg(value_name = "MODEL_OR_ASSET_TYPE")] - model_ref: String, + model_ref: ModelRef, /// Dataset repository (used only when model_ref == "dataset"). /// Per `apr-cli-pull-dataset-v1.yaml`. #[arg(value_name = "REPO")] - repo: Option, + repo: Option, /// Force re-download even if cached #[arg(long)] force: bool, @@ -570,7 +573,7 @@ pub enum Commands { /// CRUX-A-03: pin to a specific branch, tag, or git SHA on the remote /// (HuggingFace Hub). Defaults to "main" when omitted. #[arg(long, value_name = "REV")] - revision: Option, + revision: Option, /// CRUX-A-20: offline mode — forbid any outbound network I/O. /// Equivalent to APR_OFFLINE=1 or HF_HUB_OFFLINE=1 in the environment. #[arg(long)] @@ -579,11 +582,11 @@ pub enum Commands { /// multiple times; matches are unioned. fnmatch-compatible /// (`*`, `?`, `[a-z]`). No-match is fail-fast. #[arg(long, value_name = "GLOB")] - include: Vec, + include: Vec, /// (dataset mode) Output directory. Default: /// `~/.cache/aprender/datasets//`. #[arg(short = 'o', long)] - output: Option, + output: Option, }, /// Registry operations (CRUX-A-01): inspect alias map, etc. Registry { @@ -598,22 +601,22 @@ pub enum Commands { Rm { /// Model reference to remove #[arg(value_name = "MODEL")] - model_ref: String, + model_ref: ModelRef, }, /// Convert/optimize model Convert { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Quantize to format (int8, int4, fp16, q4k) #[arg(long)] - quantize: Option, + quantize: Option, /// Compress output (none, zstd, zstd-max, lz4) #[arg(long)] - compress: Option, + compress: Option, /// Output file path #[arg(short, long)] - output: PathBuf, + output: OutputPath, /// Force overwrite existing files #[arg(short, long)] force: bool, @@ -627,16 +630,16 @@ pub enum Commands { Stamp { /// Path to input .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// SPDX license identifier (e.g., Apache-2.0) #[arg(long)] - license: Option, + license: Option, /// Training-data source (e.g., huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) #[arg(long = "data-source")] - data_source: Option, + data_source: Option, /// SPDX license for data_source (e.g., Apache-2.0) #[arg(long = "data-license")] - data_license: Option, + data_license: Option, /// HuggingFace class name (e.g., Qwen2ForCausalLM, LlamaForCausalLM). /// /// PMAT-690 P0-K extension (SPEC §86): patch the upstream @@ -644,12 +647,12 @@ pub enum Commands { /// consumers (apr inspect --quality, apr pretrain --init, /// apr export → llama-cli) see the correct HF identity. #[arg(long = "hf-architecture")] - hf_architecture: Option, + hf_architecture: Option, /// HuggingFace model_type slug (e.g., qwen2, llama). /// /// PMAT-690 P0-K extension (SPEC §86). #[arg(long = "hf-model-type")] - hf_model_type: Option, + hf_model_type: Option, /// Lowercase architecture family slug (e.g., qwen2, llama). /// /// PMAT-690 P0-K extension (SPEC §86). This is the field @@ -657,7 +660,7 @@ pub enum Commands { /// patching it, pre-P0-K checkpoints with the P0-H "LlamaForCausalLM" /// fallback in this field cannot be loaded as Qwen2 inits. #[arg(long)] - architecture: Option, + architecture: Option, /// Directory containing tokenizer files (vocab.json + merges.txt /// OR tokenizer.json). When provided, embeds the vocabulary + /// BPE merges into the APR's `custom.tokenizer.vocabulary` / @@ -670,10 +673,10 @@ pub enum Commands { /// the tokenizer post-hoc so the artifact is self-contained /// for inference (the apr binary's headline use case). #[arg(long = "tokenizer", value_name = "DIR")] - tokenizer_dir: Option, + tokenizer_dir: Option, /// Output file path #[arg(short, long)] - output: PathBuf, + output: OutputPath, /// Force overwrite existing files #[arg(short, long)] force: bool, @@ -682,16 +685,16 @@ pub enum Commands { Compile { /// Input .apr model file #[arg(value_name = "FILE", required_unless_present = "list_targets")] - file: Option, + file: Option, /// Output binary path (default: derived from model name) #[arg(short, long)] - output: Option, + output: Option, /// Target triple (e.g., x86_64-unknown-linux-musl) #[arg(long)] - target: Option, + target: Option, /// Quantize weights before embedding (int8, int4, fp16) #[arg(long)] - quantize: Option, + quantize: Option, /// Release mode (optimized) #[arg(long)] release: bool, @@ -709,19 +712,19 @@ pub enum Commands { Merge { /// Model files to merge #[arg(value_name = "FILES", num_args = 2..)] - files: Vec, + files: Vec, /// Merge strategy (average, weighted, slerp, ties, dare) #[arg(long, default_value = "average")] - strategy: String, + strategy: FreeText, /// Output file path (optional in --plan mode) #[arg(short, long, required_unless_present = "plan")] - output: Option, + output: Option, /// Weights for weighted merge (comma-separated, e.g., "0.7,0.3") #[arg(long, value_delimiter = ',')] weights: Option>, /// Base model for TIES/DARE (task vectors computed as delta from base) #[arg(long)] - base_model: Option, + base_model: Option, /// DARE drop probability (default: 0.9) #[arg(long, default_value = "0.9")] drop_rate: f32, @@ -742,19 +745,19 @@ pub enum Commands { Quantize { /// Input model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Quantization scheme: int8, int4, fp16, q4k #[arg(long, short = 's', default_value = "int4")] - scheme: String, + scheme: FreeText, /// Output file path (required unless --plan) #[arg(short, long)] - output: Option, + output: Option, /// Output format override (apr, gguf, safetensors) #[arg(long)] - format: Option, + format: Option, /// Batch quantization (comma-separated schemes) #[arg(long)] - batch: Option, + batch: Option, /// Plan mode (estimate only, no execution) #[arg(long)] plan: bool, @@ -769,18 +772,21 @@ pub enum Commands { /// /// Exposes `apr` as MCP tools for Claude Code, Cursor, Cline, and other /// MCP clients. Configure via `.mcp.json` with `{"command":"apr","args":["mcp"]}`. + // #3745 S1 v1.1: prompts arrive over stdio JSON-RPC (the apr.run / apr.serve tools), not through a PromptText arg, + // so the generator is declared by marker (`apr surface` → `generates: true`). + #[command(group(batuta_common::cli_roles::ServesGeneration::group()))] Mcp {}, /// Interactive terminal UI Tui { /// Path to .apr model file #[arg(value_name = "FILE")] - file: Option, + file: Option, }, /// Model self-test: 10-stage pipeline integrity check (APR-TRACE-001) Check { /// Path to model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Disable GPU acceleration #[arg(long)] no_gpu: bool, @@ -800,19 +806,19 @@ pub enum Commands { Code { /// Path to local GGUF/APR model file (prefers .apr format) #[arg(long)] - model: Option, + model: Option, /// Project directory (loads APR.md/CLAUDE.md from this path) #[arg(long, default_value = ".")] - project: PathBuf, + project: DirPath, /// Resume previous session (optionally by ID) #[arg(long)] - resume: Option>, + resume: Option>, /// Agent manifest (advanced — overrides defaults) #[arg(long)] - manifest: Option, + manifest: Option, /// Initial prompt (non-interactive: print response and exit) #[arg(short, long)] @@ -826,7 +832,7 @@ pub enum Commands { /// — a wrong-model execution with no diagnostic — and typo'd flags /// produced no parse error at all. Options are now parsed in any /// position; a prompt that genuinely starts with `-` needs `--`. - prompt: Vec, + prompt: Vec, /// Max turns before stopping #[arg(long, default_value = "50")] @@ -838,7 +844,7 @@ pub enum Commands { /// (`§ trace_schema`). Used by `ccpa measure` to score apr-code /// against canonical Claude Code reference fixtures. #[arg(long)] - emit_trace: Option, + emit_trace: Option, /// Output format for non-interactive (`-p`) mode (PMAT-CODE-OUTPUT-FORMAT-001). /// `text` (default): plain assistant text. @@ -889,6 +895,21 @@ pub enum Commands { #[command(subcommand)] Cgp(cgp::cli::Commands), + /// Print apr's own command surface as JSON (`apr-cli-surface/v1`, #3745). + /// + /// Hidden: it is a gate input, not a user command. Every subcommand path and + /// every argument, with each argument's role read from the type that built + /// it. Release cells are derived from this output rather than from a list + /// someone typed. + #[command(hide = true)] + Surface, + + /// PTX falsification analysis and FKR test generation: analyze, gen-fkr + /// (was the `aprender-ptx-debug` binary; pure Rust, so unlike `apr ptx` + /// it needs no build feature) + #[command(subcommand, name = "ptx-debug")] + PtxDebug(trueno_ptx_debug::cli::Command), + /// Provable-contracts: validate, lint, score, kani, proof-status /// (the `pv` binary keeps shipping under its own name; this is the /// in-apr route to the same commands) @@ -912,10 +933,10 @@ pub enum DebugCommands { EmbedViz { /// Model file holding the embedding table #[arg(long, value_name = "FILE")] - model: PathBuf, + model: ModelPath, /// Embedding tensor name (default: auto-detect the known names) #[arg(long, value_name = "NAME")] - tensor: Option, + tensor: Option, /// Projection method: exact `pca`, seeded `random`, or `umap` (refused) #[arg(long, value_enum, default_value_t = EmbedProjection::Pca)] projection: EmbedProjection, @@ -928,10 +949,10 @@ pub enum DebugCommands { /// Token text, one per line, for the `token_str` column. Without it apr /// reads the GGUF vocabulary, or writes `` #[arg(long, value_name = "FILE")] - tokens: Option, + tokens: Option, /// Write the CSV here instead of stdout #[arg(short, long, value_name = "FILE")] - output: Option, + output: Option, /// Overwrite an existing --output file (refused without it) #[arg(short, long)] force: bool, diff --git a/crates/apr-cli/src/data_commands.rs b/crates/apr-cli/src/data_commands.rs index e09cabcb16..e847c9587d 100644 --- a/crates/apr-cli/src/data_commands.rs +++ b/crates/apr-cli/src/data_commands.rs @@ -54,25 +54,25 @@ pub enum DataCommands { Audit { /// Path to JSONL data file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Number of output classes (for label range validation) #[arg(long, default_value = "5")] num_classes: usize, /// Input text column name #[arg(long, default_value = "input")] - input_column: String, + input_column: FreeText, /// Label column name #[arg(long, default_value = "label")] - label_column: String, + label_column: FreeText, /// Preamble prefix to detect (e.g., "#!/") #[arg(long, default_value = "#!/")] - preamble_prefix: Option, + preamble_prefix: Option, }, /// Stratified train/val/test split preserving class proportions Split { /// Path to JSONL data file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Training set fraction #[arg(long, default_value = "0.8")] train: f64, @@ -84,22 +84,22 @@ pub enum DataCommands { test: f64, /// Label column name for stratification #[arg(long, default_value = "label")] - label_column: String, + label_column: FreeText, /// Random seed for deterministic split #[arg(long, default_value = "42")] seed: u64, /// Output directory for split files #[arg(short, long)] - output: PathBuf, + output: OutputPath, }, /// Check training data for benchmark contamination via n-gram overlap Decontaminate { /// Path to training JSONL data file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Reference benchmark JSONL files to check against #[arg(long, required = true, num_args = 1..)] - reference: Vec, + reference: Vec, /// N-gram size for overlap detection (must be >= 1) #[arg(long, default_value = "10", value_parser = parse_ngram_size)] ngram: usize, @@ -114,10 +114,10 @@ pub enum DataCommands { Dedup { /// Path to JSONL data file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Output file path for the deduplicated dataset #[arg(short, long)] - output: PathBuf, + output: OutputPath, /// Output as JSON #[arg(long)] json: bool, @@ -126,13 +126,13 @@ pub enum DataCommands { Balance { /// Path to JSONL data file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Rebalancing strategy: oversample, undersample, sqrt-inverse #[arg(long, default_value = "oversample")] - strategy: String, + strategy: FreeText, /// Label column name #[arg(long, default_value = "label")] - label_column: String, + label_column: FreeText, /// Number of classes (for sqrt-inverse weight computation) #[arg(long)] num_classes: Option, @@ -141,6 +141,6 @@ pub enum DataCommands { seed: u64, /// Output file path (required for oversample/undersample) #[arg(short, long)] - output: Option, + output: Option, }, } diff --git a/crates/apr-cli/src/dispatch.rs b/crates/apr-cli/src/dispatch.rs index dcd31a50f1..5d2e615567 100644 --- a/crates/apr-cli/src/dispatch.rs +++ b/crates/apr-cli/src/dispatch.rs @@ -5,6 +5,11 @@ fn dispatch_core_command(cli: &Cli) -> Option> { contract_pre_side_effect_classification!(); contract_pre_dispatch_completeness!(); contract_pre_output_format_fidelity!(); + // #3745 S1: the gate input, answered before any command that loads a model. + if matches!(cli.command.as_ref(), Commands::Surface) { + return Some(crate::surface::run()); + } + // Try runtime commands first (check, run, serve) if let Some(result) = dispatch_runtime_commands(cli) { return Some(result); @@ -91,6 +96,9 @@ fn dispatch_sibling_cli_commands(cli: &Cli) -> Option> { cgp::cli::dispatch(command.clone(), cli.json) .map_err(|e| CliError::ValidationFailed(format!("cgp: {e}"))), ), + Commands::PtxDebug(command) => Some(ptx_debug_result( + trueno_ptx_debug::run::run(command.clone()), + )), Commands::Pv(command) => Some( aprender_contracts_cli::dispatch(command.clone()) .map_err(|e| CliError::ValidationFailed(format!("pv: {e}"))), @@ -105,6 +113,24 @@ fn dispatch_sibling_cli_commands(cli: &Cli) -> Option> { /// signatures that #3606 is changing at the same time; the flag is still a flag to the user, and /// the guard prints `--revalidate` as its reason when it fires. (Extracted from /// `dispatch_runtime_commands` unchanged, to keep it under the complexity ratchet.) +/// Map an `apr ptx-debug` outcome onto apr's error type (#4062). +/// +/// The standalone binary exits 1/2/3 for its analysis verdict (score < 90, +/// score < `--min-score`, critical bugs). apr has one exit-code convention, so +/// every non-zero verdict becomes a validation failure (exit 5) whose message +/// names the verdict — a failing analysis must never exit 0. +fn ptx_debug_result(outcome: Result) -> Result<(), CliError> { + let verdict = match outcome { + Ok(0) => return Ok(()), + Ok(1) => "score below 90", + Ok(2) => "score below --min-score", + Ok(3) => "critical bugs present", + Ok(_) => "analysis failed", + Err(e) => return Err(CliError::ValidationFailed(format!("ptx-debug: {e}"))), + }; + Err(CliError::ValidationFailed(format!("ptx-debug: {verdict}"))) +} + fn request_f2_revalidate(revalidate: bool) { if revalidate { // SAFETY-BY-ORDER: set before any inference thread exists; the @@ -213,10 +239,10 @@ fn dispatch_runtime_commands(cli: &Cli) -> Option> { `cuda` feature, so the CUDA backend does not exist in this binary. \ Refusing to silently fall back to wgpu/CPU: that path is ~20x slower \ (~20 tok/s vs ~400) and makes any throughput measurement taken through it \ -meaningless. Rebuild the ROOT facade with CUDA: `cargo build --release \ ---features cuda` (build the root, not `-p apr-cli`: BOTH packages define a \ -binary named `apr`, and only the root's cuda = [\"cli\", \"apr-cli/cuda\"] \ -chain enables this path). To run on this build anyway, pass `--backend cpu` \ +meaningless. Rebuild with CUDA: `cargo build --release --features cuda` \ +(the root facade) or `cargo build --release -p apr-cli --features cuda` \ +(what the nightly CUDA asset builds). Both `apr` binaries run the same \ +`apr_cli::cli_main`. To run on this build anyway, pass `--backend cpu` \ or drop `--backend`." .to_string(), ))); @@ -261,11 +287,12 @@ or drop `--backend`." // GH-240: merge global --json flag into output format let effective_format = if cli.json { "json" } else { format.as_str() }; + let trace_steps = trace_steps.as_deref().map(batuta_common::cli_roles::strings); dispatch_run( source, - positional_prompt.as_ref(), + positional_prompt.as_ref().map(PromptText::as_string), input.as_deref(), - prompt.as_ref(), + prompt.as_ref().map(PromptText::as_string), *max_tokens, *stream, language.as_deref(), @@ -280,7 +307,7 @@ or drop `--backend`." *trace_payload, trace_steps.as_deref(), *trace_verbose, - trace_output.clone(), + trace_output.as_deref().map(Path::to_path_buf), trace_level.as_str(), *profile, *chat, @@ -311,14 +338,14 @@ or drop `--backend`." output_format, input_format, } => dispatch_code_command(CodeArgs { - model, + model: &model.as_deref().map(Path::to_path_buf), project, - resume, - prompt, + resume: &resume.as_ref().map(|r| r.as_ref().map(FreeText::to_string)), + prompt: &batuta_common::cli_roles::strings(prompt), print: *print, max_turns: *max_turns, - manifest, - emit_trace, + manifest: &manifest.as_deref().map(Path::to_path_buf), + emit_trace: &emit_trace.as_deref().map(Path::to_path_buf), output_format: *output_format, input_format: *input_format, }), @@ -428,13 +455,13 @@ fn dispatch_debug( }) = action { return commands::embed_viz::run(&commands::embed_viz::EmbedVizArgs { - model: model.clone(), - tensor: tensor.clone(), + model: model.to_path_buf(), + tensor: tensor.as_deref().map(str::to_string), projection: *projection, seed: *seed, limit: *limit, - tokens: tokens.clone(), - output: output.clone(), + tokens: tokens.as_deref().map(Path::to_path_buf), + output: output.as_deref().map(Path::to_path_buf), force: *force, }); } @@ -516,7 +543,7 @@ fn dispatch_inspection_commands(cli: &Cli) -> Option> { Commands::Manifest { files, output } => { // CRUX-G-05 — SHA-256 manifest of the input file set. - commands::manifest::run(files, output, cli.json) + commands::manifest::run(&batuta_common::cli_roles::path_bufs(files), output, cli.json) } Commands::Explain { code_or_file, @@ -527,8 +554,8 @@ fn dispatch_inspection_commands(cli: &Cli) -> Option> { verbose, proof_status, } => explain::run( - code_or_file.clone(), - file.clone(), + code_or_file.as_deref().map(str::to_string), + file.as_deref().map(Path::to_path_buf), tensor.as_deref(), *kernel, *json || cli.json, @@ -775,7 +802,7 @@ fn dispatch_format_commands(cli: &Cli) -> Option> { quantize.as_deref(), *strict, *preserve_q4k, - tokenizer.as_ref(), + tokenizer.as_deref().map(Path::to_path_buf).as_ref(), *enforce_provenance, *allow_no_config, cli.json, @@ -908,7 +935,7 @@ fn dispatch_model_commands(cli: &Cli) -> Option> { strategy, output.as_deref(), weights.clone(), - base_model.clone(), + base_model.as_deref().map(Path::to_path_buf), *drop_rate, *density, *seed, @@ -983,7 +1010,7 @@ fn dispatch_model_commands(cli: &Cli) -> Option> { coordinator.as_deref(), *expect_workers, *wait_gpu, - adapters, + &batuta_common::cli_roles::strings(adapters), adapters_config.as_deref(), cli.json, *experimental_mps, @@ -1073,7 +1100,7 @@ fn dispatch_model_commands(cli: &Cli) -> Option> { // full downloads in violation of the contract. Some(r) => pull::run_dataset( r, - include, + &batuta_common::cli_roles::strings(include), revision.as_deref(), output.as_deref(), *dry_run, @@ -1101,7 +1128,7 @@ fn dispatch_model_commands(cli: &Cli) -> Option> { Commands::Registry { command } => crate::commands::registry::run(command.clone()), Commands::List => pull::list(cli.json, cli.quiet), Commands::Rm { model_ref } => pull::remove(model_ref, cli.json), - Commands::Tui { file } => tui::run(file.clone()), + Commands::Tui { file } => tui::run(file.as_deref().map(Path::to_path_buf)), Commands::Mcp {} => mcp::run(), _ => return None, diff --git a/crates/apr-cli/src/dispatch_analysis.rs b/crates/apr-cli/src/dispatch_analysis.rs index 7f776f9588..b65814a1de 100644 --- a/crates/apr-cli/src/dispatch_analysis.rs +++ b/crates/apr-cli/src/dispatch_analysis.rs @@ -461,7 +461,7 @@ fn dispatch_data_command(command: &DataCommands, json: bool) -> std::result::Res ngram, threshold, json: json_flag, - } => data::run_decontaminate(file, reference, *ngram, *threshold, *json_flag || json), + } => data::run_decontaminate(file, &batuta_common::cli_roles::path_bufs(reference), *ngram, *threshold, *json_flag || json), DataCommands::Dedup { file, output, @@ -682,7 +682,7 @@ fn dispatch_analysis_commands_rest(cli: &Cli) -> Option> { *timeout, *json || cli.json, *verbose || cli.verbose, - skip.as_deref(), + skip.as_deref().map(batuta_common::cli_roles::strings).as_deref(), ) }), @@ -698,9 +698,9 @@ fn dispatch_analysis_commands_rest(cli: &Cli) -> Option> { validate, full, }) => oracle::run( - source.as_ref(), - family.as_ref(), - size.as_ref(), + source.as_ref().map(ModelRef::as_string), + family.as_ref().map(FreeText::as_string), + size.as_ref().map(FreeText::as_string), *compliance, *tensors, cli.json, @@ -896,20 +896,20 @@ fn dispatch_runs_command(command: &RunsCommands, cli: &Cli) -> std::result::Resu status, json, limit, - } => commands::runs::run_ls(dir, *global, status, *json || cli.json, *limit), + } => commands::runs::run_ls(&dir.as_deref().map(Path::to_path_buf), *global, status, *json || cli.json, *limit), RunsCommands::Show { run_id, dir, global, json, - } => commands::runs::run_show(run_id, dir, *global, *json || cli.json), + } => commands::runs::run_show(run_id, &dir.as_deref().map(Path::to_path_buf), *global, *json || cli.json), RunsCommands::Diff { run_a, run_b, dir, global, json, - } => commands::runs::run_diff(run_a, run_b, dir, *global, *json || cli.json), + } => commands::runs::run_diff(run_a, run_b, &dir.as_deref().map(Path::to_path_buf), *global, *json || cli.json), } } @@ -921,7 +921,7 @@ fn dispatch_experiment_command( ) -> std::result::Result<(), CliError> { match command { ExperimentCommands::View { db, global, json } => { - commands::experiment::experiment_view(db, *global, *json || cli.json) + commands::experiment::experiment_view(&db.as_deref().map(Path::to_path_buf), *global, *json || cli.json) } } } @@ -1087,7 +1087,7 @@ fn dispatch_train_command(command: &TrainCommands, cli: &Cli) -> std::result::Re budget_mb, dry_run, } => train::run_submit( - cluster, model, adapters, *rank, *epochs, *budget_mb, *dry_run, cli.json, + cluster, model, &batuta_common::cli_roles::strings(adapters), *rank, *epochs, *budget_mb, *dry_run, cli.json, ), TrainCommands::ClusterStatus { cluster } => train::run_cluster_status(cluster, cli.json), } @@ -1159,7 +1159,7 @@ fn dispatch_tokenize_command( estimate_only, estimate_sample_docs, } => tokenize::run_encode_corpus( - corpus.as_slice(), + batuta_common::cli_roles::path_bufs(corpus).as_slice(), tokenizer, output, *shard_tokens, @@ -1494,14 +1494,14 @@ fn dispatch_profiling_commands(cli: &Cli) -> Option> { *skip_contract, *skip_format_parity, *skip_ptx_parity, - safetensors_path.clone(), + safetensors_path.as_deref().map(Path::to_path_buf), *iterations, *warmup, *max_tokens, *json || cli.json, *verbose || cli.verbose, *min_executed, - previous_report.clone(), + previous_report.as_deref().map(Path::to_path_buf), *regression_threshold, *skip_gpu_state, *skip_metadata, @@ -1691,9 +1691,9 @@ fn dispatch_extended_command(cli: &Cli) -> Result<(), CliError> { *inspect, effective_no_gpu, *trace, - trace_steps.as_deref(), + trace_steps.as_deref().map(batuta_common::cli_roles::strings).as_deref(), *trace_verbose, - trace_output.clone(), + trace_output.as_deref().map(Path::to_path_buf), trace_level.as_str(), *profile, cli.offline, @@ -1748,7 +1748,7 @@ fn dispatch_extended_command(cli: &Cli) -> Result<(), CliError> { license, pipeline_tag, library_name.as_deref(), - tags.as_ref().map_or(&[], std::vec::Vec::as_slice), + &tags.as_deref().map(batuta_common::cli_roles::strings).unwrap_or_default(), message.as_deref(), *dry_run || *plan, cli.verbose, @@ -1800,7 +1800,7 @@ fn dispatch_extended_command(cli: &Cli) -> Result<(), CliError> { token_type_ids.as_deref(), query.as_deref(), passage.as_deref(), - passages, + &batuta_common::cli_roles::strings(passages), *sort, *top_k, vocab.as_deref(), @@ -1834,7 +1834,7 @@ fn dispatch_extended_command(cli: &Cli) -> Result<(), CliError> { json, } => commands::embed::run( model, - text, + &batuta_common::cli_roles::strings(text), text_file.as_deref(), vocab, pool, diff --git a/crates/apr-cli/src/dispatch_run.rs b/crates/apr-cli/src/dispatch_run.rs index 809ee1d37b..e4fb59954d 100644 --- a/crates/apr-cli/src/dispatch_run.rs +++ b/crates/apr-cli/src/dispatch_run.rs @@ -212,14 +212,14 @@ fn dispatch_serve_command(command: &ServeCommands, cli: &Cli) -> Result<(), CliE *no_metrics, *no_gpu, *gpu, - gpu_layers, + &gpu_layers.as_deref().map(str::to_string), *batch, *trace, trace_level, *profile, cli.verbose, backend, - otlp_endpoint, + &otlp_endpoint.as_deref().map(str::to_string), *context_length, *no_fp8_cache, *ollama_compat, @@ -296,7 +296,12 @@ fn dispatch_rosetta(action: &RosettaCommands, global_json: bool) -> Result<(), C formats, work_dir, json, - } => rosetta::run_chain(source, formats, work_dir, *json || global_json), + } => rosetta::run_chain( + source, + &batuta_common::cli_roles::strings(formats), + work_dir, + *json || global_json, + ), RosettaCommands::Verify { source, intermediate, @@ -344,8 +349,8 @@ fn dispatch_rosetta(action: &RosettaCommands, global_json: bool) -> Result<(), C json, } => rosetta::run_fingerprint( model, - model_b.as_ref().map(std::path::PathBuf::as_path), - output.as_ref().map(std::path::PathBuf::as_path), + model_b.as_deref(), + output.as_deref(), filter.as_deref(), *verbose, *json || global_json, @@ -359,8 +364,8 @@ fn dispatch_rosetta(action: &RosettaCommands, global_json: bool) -> Result<(), C json, } => rosetta::run_validate_stats( model, - reference.as_ref().map(std::path::PathBuf::as_path), - fingerprints.as_ref().map(std::path::PathBuf::as_path), + reference.as_deref(), + fingerprints.as_deref(), *threshold, *strict, *json || global_json, diff --git a/crates/apr-cli/src/extended_commands.rs b/crates/apr-cli/src/extended_commands.rs index 1be58039bd..778e4ef1f7 100644 --- a/crates/apr-cli/src/extended_commands.rs +++ b/crates/apr-cli/src/extended_commands.rs @@ -37,22 +37,26 @@ pub enum EmbedProjection { #[derive(Subcommand, Debug)] pub enum ExtendedCommands { /// Interactive chat with language model + // #3745 S1 v1.1: prompts arrive over stdin (its only PromptText is --system), not through a PromptText arg, + // so the generator is declared by marker (`apr surface` → `generates: true`). + #[command(group(batuta_common::cli_roles::ServesGeneration::group()))] + #[command(groups(batuta_common::cli_roles::SamplingArg::groups()))] Chat { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Sampling temperature (0 = greedy, higher = more random) - #[arg(long, default_value = "0.7")] + #[arg(long, default_value = "0.7", group = batuta_common::cli_roles::SamplingKind::Temperature.id())] temperature: f32, /// Nucleus sampling threshold - #[arg(long, default_value = "0.9")] + #[arg(long, default_value = "0.9", group = batuta_common::cli_roles::SamplingKind::TopP.id())] top_p: f32, /// Maximum tokens to generate per response #[arg(long, default_value = "512")] max_tokens: usize, /// System prompt to set model behavior #[arg(long)] - system: Option, + system: Option, /// Show inspection info (top-k probs, tokens/sec) #[arg(long)] inspect: bool, @@ -67,13 +71,13 @@ pub enum ExtendedCommands { trace: bool, /// Trace specific steps only (comma-separated) #[arg(long, value_delimiter = ',')] - trace_steps: Option>, + trace_steps: Option>, /// Verbose tracing #[arg(long)] trace_verbose: bool, /// Save trace output to JSON file #[arg(long, value_name = "FILE")] - trace_output: Option, + trace_output: Option, /// Trace detail level (none, basic, layer, payload) #[arg(long, value_name = "LEVEL", default_value = "basic", value_parser = TRACE_LEVEL_VALUES)] trace_level: String, @@ -88,7 +92,7 @@ pub enum ExtendedCommands { Bench { /// Path to model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Number of warmup iterations #[arg(long, default_value = "3")] warmup: usize, @@ -100,13 +104,13 @@ pub enum ExtendedCommands { max_tokens: usize, /// Test prompt #[arg(long)] - prompt: Option, + prompt: Option, /// Use realizar for fast inference (vs aprender baseline) #[arg(long)] fast: bool, /// Benchmark specific brick #[arg(long)] - brick: Option, + brick: Option, /// Comma-separated latency percentile points for JSON output /// (CRUX-E-07). Default: `50,95,99`. Values must be in (0, 100]. #[arg( @@ -121,13 +125,13 @@ pub enum ExtendedCommands { Eval { /// Path to model file or checkpoint directory #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Dataset: wikitext-2, lambada, or custom #[arg(long, default_value = "wikitext-2")] - dataset: String, + dataset: FreeText, /// Custom text (when dataset=custom) #[arg(long)] - text: Option, + text: Option, /// Maximum tokens to evaluate #[arg(long, default_value = "512")] max_tokens: usize, @@ -137,13 +141,13 @@ pub enum ExtendedCommands { threshold: f32, /// Task type: omit for perplexity, "classify" for classification eval #[arg(long)] - task: Option, + task: Option, /// Test data file (JSONL) for classification evaluation #[arg(long, value_name = "FILE")] - data: Option, + data: Option, /// Model size hint: "0.5B", "tiny" (for classification eval) #[arg(long)] - model_size: Option, + model_size: Option, /// Number of output classes (default: 5) #[arg(long, default_value = "5")] num_classes: usize, @@ -165,16 +169,16 @@ pub enum ExtendedCommands { Profile { /// Path to model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Layer-by-layer granular analysis #[arg(long)] granular: bool, /// Output format (human, json, flamegraph) #[arg(long, default_value = "human")] - format: String, + format: FreeText, /// Focus on specific operation #[arg(long)] - focus: Option, + focus: Option, /// Detect naive implementations #[arg(long)] detect_naive: bool, @@ -184,7 +188,7 @@ pub enum ExtendedCommands { threshold: f64, /// [NOT IMPLEMENTED — accepted and ignored] Compare against HuggingFace baseline #[arg(long)] - compare_hf: Option, + compare_hf: Option, /// [NOT IMPLEMENTED — accepted and ignored] Measure energy consumption (requires RAPL) #[arg(long)] energy: bool, @@ -199,7 +203,7 @@ pub enum ExtendedCommands { fail_on_naive: bool, /// Output file path for flamegraph SVG (GH-174, PMAT-182) #[arg(long, short = 'o')] - output: Option, + output: Option, // PMAT-192: CI Assertion Mode (GH-180) /// Enable CI mode with assertion checks (exits 1 on failure) @@ -232,13 +236,13 @@ pub enum ExtendedCommands { no_gpu: bool, /// Compare against another model format (F-PROFILE-011) #[arg(long, value_name = "FILE")] - compare: Option, + compare: Option, }, /// Falsifiable QA checklist for model releases Qa { /// Path to model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Minimum throughput threshold in tok/s #[arg(long, value_name = "TPS", value_parser = commands::threshold_arg::parse_tolerance)] @@ -274,7 +278,7 @@ pub enum ExtendedCommands { skip_ptx_parity: bool, /// SafeTensors model path for cross-format parity test (F-QUAL-032) #[arg(long, value_name = "PATH")] - safetensors_path: Option, + safetensors_path: Option, /// Number of benchmark iterations #[arg(long, default_value = "10")] iterations: usize, @@ -295,7 +299,7 @@ pub enum ExtendedCommands { min_executed: Option, /// Previous QA report for regression detection #[arg(long, value_name = "FILE")] - previous_report: Option, + previous_report: Option, /// Maximum allowed performance regression ratio (default: 0.10 = 10%) #[arg(long, value_name = "RATIO", value_parser = commands::threshold_arg::parse_fraction)] @@ -317,10 +321,10 @@ pub enum ExtendedCommands { Parity { /// Path to GGUF model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Prompt text (default: "What is 2+2?") #[arg(short, long, default_value = "What is 2+2?")] - prompt: String, + prompt: PromptText, /// Assert parity (exit non-zero on divergence) #[arg(long)] assert: bool, @@ -330,7 +334,7 @@ pub enum ExtendedCommands { per_op: bool, /// Where --per-op writes the two APRT trees (default: a temp dir) #[arg(long, value_name = "DIR")] - out: Option, + out: Option, /// Cosine threshold for --per-op (basis: evidence/parity/thresholds.yaml) #[arg(long, default_value_t = 0.98)] threshold: f32, @@ -340,13 +344,13 @@ pub enum ExtendedCommands { PtxMap { /// Path to GGUF model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Filter to specific kernel (e.g., --kernel Q4KGemv) #[arg(long)] - kernel: Option, + kernel: Option, /// Reverse lookup: kernel name -> which layers/steps use it #[arg(long)] - reverse: Option, + reverse: Option, /// Output as JSON #[arg(long)] json: bool, @@ -366,16 +370,16 @@ pub enum ExtendedCommands { not(feature = "trueno-explain"), command( name = "ptx", - about = "PTX analysis and bug detection [unavailable in this build: cargo install aprender --features ptx]" + about = "PTX analysis and bug detection [unavailable in this build: cargo install aprender --features ptx; `apr ptx-debug` is always built]" ) )] Ptx { /// Path to a PTX source file #[arg(value_name = "FILE")] - file: Option, + file: Option, /// Analyze a named kernel from trueno-gpu #[arg(long, short)] - kernel: Option, + kernel: Option, /// Strict mode (no performance whitelist) #[arg(long)] strict: bool, @@ -394,10 +398,10 @@ pub enum ExtendedCommands { Tune { /// Path to model file (optional if using --model) #[arg(value_name = "FILE")] - file: Option, + file: Option, /// Tuning method: auto, full, lora, qlora #[arg(long, short = 'm', default_value = "auto")] - method: String, + method: FreeText, /// LoRA rank (default: auto-selected) #[arg(long, short = 'r')] rank: Option, @@ -409,56 +413,56 @@ pub enum ExtendedCommands { plan: bool, /// Model size for planning (e.g., "7B", "1.5B") #[arg(long, value_name = "SIZE")] - model: Option, + model: Option, /// Freeze base model weights #[arg(long)] freeze_base: bool, /// Training data file (JSONL format) #[arg(long, value_name = "FILE")] - train_data: Option, + train_data: Option, /// Output as JSON (for CI integration) #[arg(long)] json: bool, /// Task type for HPO: classify (SPEC-TUNE-2026-001) #[arg(long)] - task: Option, + task: Option, /// Number of HPO trials (default: 10) #[arg(long, default_value = "10")] budget: usize, /// HPO search strategy: tpe, grid, random #[arg(long, default_value = "tpe")] - strategy: String, + strategy: FreeText, /// HPO scheduler: asha, median, none #[arg(long, default_value = "asha")] - scheduler: String, + scheduler: FreeText, /// Scout mode: 1 epoch per trial for fast exploration #[arg(long)] scout: bool, /// Training data file for HPO (JSONL format) #[arg(long, value_name = "FILE")] - data: Option, + data: Option, /// Number of output classes for classification #[arg(long, default_value = "5")] num_classes: usize, /// Model size hint for HPO (e.g., "0.5B", "1.5B") #[arg(long)] - model_size: Option, + model_size: Option, /// Warm-start from scout phase results directory #[arg(long, value_name = "DIR")] - from_scout: Option, + from_scout: Option, /// Maximum epochs per trial (full mode, default: 20) #[arg(long, default_value = "20")] max_epochs: usize, /// Maximum wall-clock time (e.g., "8h", "30m") #[arg(long)] - time_limit: Option, + time_limit: Option, }, /// Attach live TUI to a running training session #[cfg(feature = "training")] Monitor { /// Experiment output directory (same as finetune -o) #[arg(value_name = "DIR")] - dir: Option, + dir: Option, /// Refresh interval in milliseconds #[arg(long, default_value = "500")] refresh_ms: u64, @@ -470,7 +474,7 @@ pub enum ExtendedCommands { json: bool, /// Output format: tui (default), json, text #[arg(long, default_value = "tui")] - format: String, + format: FreeText, }, /// List, show, and compare training experiment runs #[cfg(feature = "training")] @@ -488,13 +492,13 @@ pub enum ExtendedCommands { Cbtop { /// Model name (e.g., qwen2.5-coder-1.5b) #[arg(long)] - model: Option, + model: Option, /// Attach to running realizar process #[arg(long)] - attach: Option, + attach: Option, /// Path to GGUF model file for real profiling #[arg(long, value_name = "MODEL")] - model_path: Option, + model_path: Option, /// Run in headless mode (no TUI, for CI/automation) #[arg(long)] headless: bool, @@ -503,7 +507,7 @@ pub enum ExtendedCommands { json: bool, /// Output file path (requires --headless) #[arg(long, value_name = "FILE", requires = "headless")] - output: Option, + output: Option, /// CI mode: exit non-zero if thresholds are not met or the report status is FAIL #[arg(long)] ci: bool, @@ -528,7 +532,7 @@ pub enum ExtendedCommands { speculation_k: usize, /// PAR-099: Path to draft model for speculative decoding #[arg(long, value_name = "DRAFT_MODEL")] - draft_model: Option, + draft_model: Option, /// PAR-102: Number of concurrent requests #[arg(long, default_value = "1")] concurrent: usize, @@ -573,13 +577,13 @@ pub enum ExtendedCommands { CompareHf { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// HuggingFace repo ID (e.g., openai/whisper-tiny) #[arg(long)] - hf: String, + hf: ModelRef, /// Filter tensors by name pattern #[arg(long)] - tensor: Option, + tensor: Option, /// Comparison threshold (default: 1e-5) #[arg(long, default_value = "1e-5", value_parser = commands::threshold_arg::parse_tolerance)] @@ -597,10 +601,10 @@ pub enum ExtendedCommands { Hex { /// Path to model file (APR, GGUF, or SafeTensors) #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Filter tensors by name pattern #[arg(long)] - tensor: Option, + tensor: Option, /// Limit bytes/values to display #[arg(long, default_value = "64")] limit: usize, @@ -633,13 +637,13 @@ pub enum ExtendedCommands { raw: bool, /// Start at byte offset (supports 0x prefix for hex) #[arg(long, default_value = "0")] - offset: String, + offset: FreeText, /// Bytes per row for raw output (default: 16) #[arg(long, default_value = "16")] width: usize, /// Slice range for partial tensor reads (e.g., 0:3 for first 3 elements) #[arg(long)] - slice: Option, + slice: Option, }, /// Backend discovery: probe, enumerate, print — every kind is a line (PP-066 R-0) #[cfg(feature = "inference")] @@ -652,10 +656,10 @@ pub enum ExtendedCommands { Tree { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Filter by component pattern #[arg(long)] - filter: Option, + filter: Option, /// Output format: ascii, dot, mermaid, json /// /// #2394 finding 15: this was a `String` that the dispatcher parsed @@ -677,13 +681,13 @@ pub enum ExtendedCommands { Flow { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Filter by layer pattern #[arg(long)] - layer: Option, + layer: Option, /// Component to visualize: full, encoder, decoder, etc. #[arg(long, default_value = "full")] - component: String, + component: FreeText, /// Verbose output with statistics #[arg(short, long)] verbose: bool, @@ -695,10 +699,10 @@ pub enum ExtendedCommands { Qualify { /// Path to model file (APR, GGUF, or SafeTensors) #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Testing tier: smoke (Phase 1), standard (+contracts), full (+playbook) #[arg(long, default_value = "smoke")] - tier: String, + tier: FreeText, /// Timeout per gate in seconds #[arg(long, default_value = "120")] timeout: u64, @@ -710,7 +714,7 @@ pub enum ExtendedCommands { verbose: bool, /// Skip specific gates (comma-separated) #[arg(long, value_delimiter = ',')] - skip: Option>, + skip: Option>, }, /// Training pipeline (plan/apply) — forjar-style pre-flight validation #[cfg(feature = "training")] @@ -729,13 +733,13 @@ pub enum ExtendedCommands { Pretrain { /// Dataset path (tokenized shard index or raw corpus). #[arg(long, value_name = "PATH")] - dataset: PathBuf, + dataset: InputFile, /// Tokenizer directory (vocab.json + merges.txt). #[arg(long, value_name = "DIR")] - tokenizer: PathBuf, + tokenizer: InputFile, /// Run output directory — checkpoints + metadata go to `{run_dir}/ckpt/`. #[arg(long, value_name = "DIR")] - run_dir: PathBuf, + run_dir: OutputPath, /// Training regime — finetune (MODEL-1) or from-scratch (MODEL-2 cold start). /// Per contract training-loop-pretrain-v1 §hyperparameter_defaults, /// this atomically flips (regime, lr_max, warmup_steps, target_val_loss) @@ -783,7 +787,7 @@ pub enum ExtendedCommands { /// spelling that may fall back silently — all other values /// hard-fail on missing runtime per GATE-GPUTRAIN-002). #[arg(long, default_value = "auto")] - device: String, + device: FreeText, /// Initial weights from a pretrained APR file /// (contract `apr-pretrain-from-init-v1`). Per spec §49's /// MODEL-2 pretrained-init pivot: when present, load weights @@ -793,7 +797,7 @@ pub enum ExtendedCommands { /// corrupted, or arch-mismatched APR files exit non-zero /// before step 1 (no silent random-init fallback). #[arg(long, value_name = "PATH")] - init: Option, + init: Option, /// SPEC §83 P0-J: bypass the Chinchilla compute-optimal hard /// gate (`chinchilla-gate-v1`). Default is fail-fast when /// D/N < 10× (severely under-provisioned per Hoffmann et al. @@ -822,7 +826,7 @@ pub enum ExtendedCommands { /// When omitted, falls back to the historical "first 16 /// batches of --dataset" behaviour for backwards compatibility. #[arg(long, value_name = "DIR")] - val_shard: Option, + val_shard: Option, }, /// Tokenizer training pipeline (plan/apply) — BPE vocabulary learning Tokenize { @@ -843,13 +847,13 @@ pub enum ExtendedCommands { Diagnose { /// Path to checkpoint directory #[arg(value_name = "CHECKPOINT_DIR")] - checkpoint_dir: PathBuf, + checkpoint_dir: ModelPath, /// Test data file (JSONL) for evaluation #[arg(long, value_name = "FILE")] - data: Option, + data: Option, /// Model size hint: "0.5B", "tiny" #[arg(long)] - model_size: Option, + model_size: Option, /// Number of output classes (default: 5) #[arg(long, default_value = "5")] num_classes: usize, @@ -858,7 +862,7 @@ pub enum ExtendedCommands { OllamaChatLint { /// Path to captured /api/chat response (JSON object, or NDJSON if --stream) #[arg(long, value_name = "FILE")] - response_file: PathBuf, + response_file: InputFile, /// Treat input as NDJSON stream (one frame per line) #[arg(long)] stream: bool, @@ -867,12 +871,12 @@ pub enum ExtendedCommands { OllamaToolsLint { /// Path to captured /api/chat response (JSON object, or NDJSON if --stream) #[arg(long, value_name = "FILE")] - response_file: PathBuf, + response_file: InputFile, /// Captured request JSON, required unless --stream — supplies the /// tool-name allowlist (every called tool name must appear in /// request.tools[*].function.name) #[arg(long, value_name = "FILE")] - request_file: Option, + request_file: Option, /// Treat input as NDJSON stream (one frame per line) #[arg(long)] stream: bool, @@ -881,46 +885,46 @@ pub enum ExtendedCommands { DrySamplingLint { /// Path to observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured AWQ quality/compression/flags observation (CRUX-B-08) AwqLint { /// Path to captured AWQ observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured FP8 (E4M3) round-trip + SM-capability observation (CRUX-B-11) Fp8Lint { /// Path to captured observation JSON (frobenius, capability blocks) #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured NF4 codebook/roundtrip/storage/parity observation (CRUX-B-10) Nf4Lint { /// Path to captured NF4 observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured GPTQ compression/cosine/flags observation (CRUX-B-09) GptqLint { /// Path to captured GPTQ observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured CUDA OOM postmortem report (CRUX-F-13) OomLint { /// Path to captured OOM postmortem JSON (e.g. /tmp/apr-oom-.json) #[arg(long, value_name = "FILE")] - report_file: PathBuf, + report_file: InputFile, /// Optional captured stderr log to verify the OOM_REPORT breadcrumb #[arg(long, value_name = "FILE")] - stderr_file: Option, + stderr_file: Option, }, /// Lint a captured NCCL failure-diagnostics JSON from stderr (CRUX-F-15) NcclDiagLint { /// Path to captured stderr JSON diagnostic #[arg(long, value_name = "FILE")] - diag_file: PathBuf, + diag_file: InputFile, /// Optional observed exit code (gate: >= 128 = NCCL class) #[arg(long, value_name = "I32")] exit_code: Option, @@ -932,7 +936,7 @@ pub enum ExtendedCommands { ReactTraceLint { /// Path to captured trace JSON #[arg(long, value_name = "FILE")] - trace_file: PathBuf, + trace_file: InputFile, /// Optional max_iterations budget the trace was produced under #[arg(long, value_name = "N")] max_iterations: Option, @@ -944,10 +948,10 @@ pub enum ExtendedCommands { HangTraceLint { /// Path to the captured trace directory #[arg(long, value_name = "DIR")] - trace_dir: PathBuf, + trace_dir: DirPath, /// Inspection mode: `timeout` (expects per-rank dumps) or `success` (expects empty dir) #[arg(long, value_name = "MODE", default_value = "timeout")] - mode: String, + mode: FreeText, /// Expected world_size when mode=timeout (number of rank{N}.py.txt files) #[arg(long, value_name = "N", default_value_t = 2)] world_size: usize, @@ -962,10 +966,10 @@ pub enum ExtendedCommands { DdpMetricsLint { /// Path to N=1 metrics JSON #[arg(long, value_name = "FILE")] - metrics_1gpu_file: PathBuf, + metrics_1gpu_file: InputFile, /// Path to N=world_size metrics JSON #[arg(long, value_name = "FILE")] - metrics_ngpu_file: PathBuf, + metrics_ngpu_file: InputFile, /// World size used for --metrics-ngpu-file run (>= 2) #[arg(long, value_name = "N")] world_size: i64, @@ -993,7 +997,7 @@ pub enum ExtendedCommands { AudioInspectLint { /// Path to the JSON body written by `apr dataset audio-inspect --format json` #[arg(long, value_name = "FILE")] - json_file: PathBuf, + json_file: InputFile, /// Optional expected sample_rate (typically the `--resample-to` arg) #[arg(long, value_name = "U32")] expected_sample_rate: Option, @@ -1007,16 +1011,16 @@ pub enum ExtendedCommands { /// Parity JSON body (`max_abs_diff`, `cosine_sim`), as written by /// `apr kernel parity --json` #[arg(long, value_name = "FILE")] - parity_file: Option, + parity_file: Option, /// Provenance JSON body (`attn_impl`, `kernel_source`, `fallback`). /// `apr kernel parity --json` writes both gates' fields into one body, /// so the same file may be passed here and to --parity-file #[arg(long, value_name = "FILE")] - provenance_file: Option, + provenance_file: Option, /// head_dim refusal JSON, as written by /// `apr kernel parity --impl flash2 --head-dim 96 --json` (which exits non-zero) #[arg(long, value_name = "FILE")] - head_dim_error_file: Option, + head_dim_error_file: Option, /// Max absolute diff tolerance (default 5e-3, FlashAttention-2 bound) #[arg(long, value_name = "F", default_value_t = 5e-3, value_parser = commands::threshold_arg::parse_tolerance)] @@ -1030,10 +1034,10 @@ pub enum ExtendedCommands { AttnVizLint { /// Path to attention dump in JSON form (4-D [layers][heads][rows][cols] floats) #[arg(long, value_name = "FILE")] - attn_file: Option, + attn_file: Option, /// Path to HTML heatmap output #[arg(long, value_name = "FILE")] - html_file: Option, + html_file: Option, /// Minimum , + error_file: Option, /// Externally captured check-finite layer-coverage JSON #[arg(long, value_name = "FILE")] - list_file: Option, + list_file: Option, /// Minimum layer-coverage count when `--list-file` is supplied (default 100) #[arg(long, value_name = "N", default_value_t = 100)] min_layers: usize, @@ -1063,19 +1067,19 @@ pub enum ExtendedCommands { EmbedVizLint { /// Path to the `token_id,token_str,x,y` CSV written by `apr debug embed-viz` #[arg(long, value_name = "FILE")] - csv_file: PathBuf, + csv_file: InputFile, /// Expected row count == vocab_size (optional) #[arg(long, value_name = "N")] expected_vocab_size: Option, /// Second CSV from a rerun at the same --seed, for the determinism gate (optional) #[arg(long, value_name = "FILE")] - csv_file_b: Option, + csv_file_b: Option, }, /// Lint an externally captured token-selection JSONL trace (CRUX-F-19 — no apr producer yet) ExplainTokenLint { /// Path to captured JSONL body (one sampled-token record per line) #[arg(long, value_name = "FILE")] - jsonl_file: PathBuf, + jsonl_file: InputFile, /// Tolerance for `Σ post_prob ≈ 1.0` (default 1e-5) #[arg(long, value_name = "F64", default_value_t = 1e-5, value_parser = commands::threshold_arg::parse_tolerance)] @@ -1088,13 +1092,13 @@ pub enum ExtendedCommands { GpuMemtraceLint { /// Path to an externally captured GPU-memory Chrome Trace JSON (no apr producer yet) #[arg(long, value_name = "FILE")] - trace_file: PathBuf, + trace_file: InputFile, }, /// Lint a captured KV-cache utilization timeline (CRUX-F-06) KvTimelineLint { /// Path to an externally captured KV-cache timeline JSON body (no apr producer yet) #[arg(long, value_name = "FILE")] - timeline_file: PathBuf, + timeline_file: InputFile, /// Preemption threshold (default 0.95, vLLM canonical) #[arg(long, value_name = "FRACTION", default_value_t = 0.95, value_parser = commands::threshold_arg::parse_fraction)] @@ -1107,7 +1111,7 @@ pub enum ExtendedCommands { OtlpLint { /// Path to captured OTLP/JSON export body #[arg(long, value_name = "FILE")] - otlp_file: PathBuf, + otlp_file: InputFile, /// Require at least one `apr.inference` span to be present #[arg(long)] require_apr_span: bool, @@ -1116,16 +1120,16 @@ pub enum ExtendedCommands { require_genai_attrs: bool, /// Verify W3C trace-context propagation: expect this 32-hex traceId #[arg(long, value_name = "HEX32")] - expect_trace_id: Option, + expect_trace_id: Option, }, /// Lint a captured Prometheus /metrics response (CRUX-K-07) PrometheusLint { /// Path to captured /metrics response body (text/plain; version=0.0.4) #[arg(long, value_name = "FILE")] - metrics_file: PathBuf, + metrics_file: InputFile, /// Optional captured Content-Type header to verify against version=0.0.4 #[arg(long, value_name = "HEADER")] - content_type: Option, + content_type: Option, /// Require the K-07 metric set (apr_num_requests_running, ...) to be present #[arg(long)] require_k07_metrics: bool, @@ -1134,26 +1138,26 @@ pub enum ExtendedCommands { ToolUseLint { /// Path to captured OpenAI tool-use response JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a GBNF grammar-constrained observation (CRUX-C-10) GbnfLint { /// Path to captured GBNF observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a typical-p sampling observation (CRUX-C-22) TypicalPLint { /// Path to captured typical-p observation JSON, with any of the /// sections range/identity/mass/sort/renorm #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Gradient-norm telemetry analysis (CRUX-F-09) GradNorm { /// Path to JSON file of per-step grad-norm records #[arg(long, value_name = "FILE")] - history_file: PathBuf, + history_file: InputFile, /// Maximum allowed clipped grad-norm (for cap-violation check) #[arg(long, value_name = "M", value_parser = commands::threshold_arg::parse_tolerance)] @@ -1170,66 +1174,66 @@ pub enum ExtendedCommands { RegistryQuotaLint { /// Path to captured quota/atomic/ceiling observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured imatrix calibration observation (CRUX-B-07) ImatrixLint { /// Path to captured imatrix observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured /v1/embeddings observation (CRUX-C-13) EmbeddingsLint { /// Path to captured /v1/embeddings observation JSON, with any of the /// sections shape/determinism/usage/flag #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured Hub+local unified-search merge observation (CRUX-A-23) UnifiedSearchLint { /// Path to captured unified-search observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured `apr rm` / externally captured gc blob-GC observation (CRUX-A-25) RmGcLint { /// Path to captured rm/gc observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Lint a captured APR_MODELS shared-cache observation (CRUX-A-21) SharedCacheLint { /// Path to captured dedup/permission observation JSON #[arg(long, value_name = "FILE")] - observation_file: PathBuf, + observation_file: InputFile, }, /// Perplexity classifier (CRUX-E-02) Ppl { /// JSON file containing an array of per-token natural-log /// probabilities (e.g. `[-1.2, -0.5, -2.1, ...]`). Required. #[arg(long, value_name = "FILE")] - log_probs_file: PathBuf, + log_probs_file: InputFile, }, /// Validate dequant→requant metadata preservation (CRUX-B-19) QuantPreservationLint { /// Reference GGUF (pre-roundtrip) #[arg(long, value_name = "REF.gguf")] - reference: PathBuf, + reference: ModelPath, /// Requantized GGUF (post-roundtrip) #[arg(long, value_name = "REQ.gguf")] - requant: PathBuf, + requant: ModelPath, }, /// Split a safetensors file into shards + weight-map index (CRUX-B-05) Shard { /// Single-file safetensors model to split #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Maximum size of each shard (e.g. 5GB, 500MB, 1.5GiB) #[arg(long, value_name = "SIZE", default_value = "5GB")] - max_shard_size: String, + max_shard_size: FreeText, /// Output directory for shards + model.safetensors.index.json #[arg(short, long, value_name = "DIR")] - output: PathBuf, + output: OutputPath, /// #2392: Overwrite an existing shard set in the output directory #[arg(short, long)] force: bool, @@ -1238,10 +1242,10 @@ pub enum ExtendedCommands { Unshard { /// Sharded directory containing model.safetensors.index.json #[arg(value_name = "DIR")] - input: PathBuf, + input: ModelPath, /// Output single-file safetensors path #[arg(short, long, value_name = "FILE")] - output: PathBuf, + output: OutputPath, /// #2392: Overwrite an existing output file (refused without it) #[arg(short, long)] force: bool, @@ -1262,26 +1266,26 @@ pub enum ExtendedCommands { Rerank { /// Path to the APR file containing the cross-encoder weights. #[arg(value_name = "MODEL")] - model: PathBuf, + model: ModelPath, /// Pre-tokenised input ids (comma-separated `u32`s). Mutually /// exclusive with `--query`+`--passage`+`--vocab` (Phase 3b). /// Example: `--input-ids 101,2024,102,3456,102` for `[CLS] q [SEP] p [SEP]`. #[arg(long, value_name = "IDS")] - input_ids: Option, + input_ids: Option, /// Pre-tokenised token-type ids (comma-separated `u32`s). /// Same length as `--input-ids`. 0 for query side, 1 for passage. #[arg(long, value_name = "IDS")] - token_type_ids: Option, + token_type_ids: Option, /// Phase 3b — query text. Pair with `--passage` + `--vocab` to enable /// in-process WordPiece tokenisation. The tokeniser builds /// `[CLS] query [SEP] passage [SEP]` with `token_type_ids = 0` for /// the query side and `1` for the passage side. #[arg(long, value_name = "TEXT")] - query: Option, + query: Option, /// Phase 3b — passage text. Required when `--query` is supplied /// in single-pair mode (use `--passages` for batch ranking). #[arg(long, value_name = "TEXT")] - passage: Option, + passage: Option, /// Phase 5 — batch ranking mode (#326). Passage candidates to /// score against `--query`. May be supplied multiple times: /// `apr rerank model.apr --query "..." --passages "p1" --passages "p2"`. @@ -1290,7 +1294,7 @@ pub enum ExtendedCommands { /// of `{passage, logit, score}` objects sorted by descending /// score when `--sort` is set. #[arg(long, value_name = "TEXT")] - passages: Vec, + passages: Vec, /// Phase 5 — sort batch output by descending score (highest /// relevance first). Only meaningful with `--passages` and /// `--json`. Default: preserve input order. @@ -1305,7 +1309,7 @@ pub enum ExtendedCommands { /// Must contain entries for `[CLS]`, `[SEP]`, and `[UNK]`. /// Phase 4 accepts HuggingFace `tokenizer.json` (extension-detected). #[arg(long, value_name = "FILE")] - vocab: Option, + vocab: Option, /// Override hidden_dim (default: 384 / MiniLM-L-6). #[arg(long, default_value_t = 384)] hidden_dim: usize, @@ -1367,10 +1371,10 @@ pub enum ExtendedCommands { Embed { /// Path to the APR file containing the encoder weights (BertModel). #[arg(value_name = "MODEL")] - model: PathBuf, + model: ModelPath, /// Text to encode. Repeatable: `apr embed model.apr --text "a" --text "b" --vocab tok.json`. #[arg(long, value_name = "TEXT")] - text: Vec, + text: Vec, /// Phase 7 (GH-326) — read texts from a file, one per line. /// Concatenated with `--text` inputs in order: `--text` first, /// then `--text-file` rows. Blank lines and lines starting @@ -1378,14 +1382,14 @@ pub enum ExtendedCommands { /// retrieval where the second-stage rerank candidate set /// (50-100 documents) is the embed input. #[arg(long, value_name = "FILE")] - text_file: Option, + text_file: Option, /// Path to a WordPiece `vocab.txt` or HF `tokenizer.json`. #[arg(long, value_name = "FILE")] - vocab: PathBuf, + vocab: InputFile, /// Pooling strategy (`cls` or `mean`). Default: `mean` /// (matches sentence-transformers convention). #[arg(long, default_value = "mean")] - pool: String, + pool: FreeText, /// L2-normalise the output embedding. Default: true (matches /// sentence-transformers convention). Pass `--normalize false` /// to keep raw magnitudes. @@ -1450,14 +1454,14 @@ pub enum DatasetCommands { AudioInspect { /// Path to the .wav file to decode #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Output format: `json` for the lint-readable body, `text` for humans #[arg(long, value_name = "FORMAT", default_value = "text", value_parser = ["json", "text"])] format: String, /// Write the observation here instead of stdout #[arg(short, long, value_name = "FILE")] - output: Option, + output: Option, /// Overwrite an existing --output file (refused without it) #[arg(short, long)] force: bool, @@ -1505,7 +1509,7 @@ pub enum KernelCommands { json: bool, /// Write the observation here instead of stdout #[arg(short, long, value_name = "FILE")] - output: Option, + output: Option, /// Overwrite an existing --output file (refused without it) #[arg(short, long)] force: bool, @@ -1520,13 +1524,13 @@ pub enum RunsCommands { Ls { /// Directory to scan for experiments (default: current dir) #[arg(long, value_name = "DIR")] - dir: Option, + dir: Option, /// Read from global experiment registry (~/.entrenar/experiments.db) #[arg(long)] global: bool, /// Filter by status: all, pending, running, completed, failed, cancelled #[arg(long, default_value = "all")] - status: String, + status: FreeText, /// Output as JSON #[arg(long)] json: bool, @@ -1538,10 +1542,10 @@ pub enum RunsCommands { Show { /// Run ID #[arg(value_name = "RUN_ID")] - run_id: String, + run_id: FreeText, /// Directory containing experiment DB #[arg(long, value_name = "DIR")] - dir: Option, + dir: Option, /// Read from global registry #[arg(long)] global: bool, @@ -1553,13 +1557,13 @@ pub enum RunsCommands { Diff { /// First run ID #[arg(value_name = "RUN_A")] - run_a: String, + run_a: FreeText, /// Second run ID #[arg(value_name = "RUN_B")] - run_b: String, + run_b: FreeText, /// Directory containing experiment DB #[arg(long, value_name = "DIR")] - dir: Option, + dir: Option, /// Read from global registry #[arg(long)] global: bool, @@ -1577,7 +1581,7 @@ pub enum ExperimentCommands { View { /// Path to experiment database file #[arg(long, value_name = "FILE")] - db: Option, + db: Option, /// Read from global experiment registry (~/.entrenar/experiments.db) #[arg(long)] global: bool, @@ -1599,10 +1603,10 @@ pub enum ModelfileSubcommand { Parse { /// Path to the Modelfile #[arg(value_name = "FILE")] - file: PathBuf, + file: ConfigPath, /// Output format: `json` or `human` #[arg(long, default_value = "json")] - format: String, + format: FreeText, }, } @@ -1632,19 +1636,19 @@ pub enum TestSubcommand { Tensor { /// Path to .apr model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Output directory for test artifacts #[arg(short, long, default_value = "./probar-export")] - output: PathBuf, + output: OutputPath, /// Export format: json, png, or both #[arg(long, default_value = "both")] - format: String, + format: FreeText, /// Golden reference directory for comparison #[arg(long)] - golden: Option, + golden: Option, /// Filter layers by name pattern #[arg(long)] - layer: Option, + layer: Option, /// Exit non-zero on golden divergence (CI mode, PMAT-481) #[arg(long)] assert: bool, @@ -1680,15 +1684,15 @@ pub enum LlmSubcommand { Bench { /// Endpoint to measure. #[arg(short, long, default_value = "http://127.0.0.1:8080")] - url: String, + url: FreeText, /// Model name sent in the request body. Most OpenAI-compatible /// servers ignore it; `apr serve` and vLLM do not. #[arg(short, long, default_value = "default")] - model: String, + model: FreeText, /// Command that starts the runtime. Omit to measure something already /// running. #[arg(long)] - start: Option, + start: Option, /// Seconds to wait for the endpoint to become healthy. #[arg(long, default_value = "120")] health_timeout: u64, @@ -1713,16 +1717,16 @@ pub enum LlmSubcommand { cooldown: u64, /// Label recorded in the report, e.g. `apr-cuda` or `llamacpp-39173bcac`. #[arg(long, default_value = "apr")] - runtime_name: String, + runtime_name: FreeText, /// Prior report to compare against. #[arg(long)] - baseline: Option, + baseline: Option, /// Fractional regression that fails the run, e.g. 0.10 for 10%. #[arg(long)] fail_on_regression: Option, /// Write the JSON report here. #[arg(short, long)] - output: Option, + output: Option, /// Use streaming responses (measures TTFT and TPOT). #[arg(long)] stream: bool, @@ -1734,7 +1738,7 @@ pub enum LlmSubcommand { /// knob with a recorded value rather than a constant buried in a /// helper. #[arg(long, default_value = "medium")] - profile: String, + profile: FreeText, /// Prompt corpus, JSONL: one JSON object per line, `{"prompt": "..."}`, /// with optional `max_tokens`, `temperature`, `seed`, `ignore_eos`. /// Overrides `--profile`. @@ -1749,7 +1753,7 @@ pub enum LlmSubcommand { /// so an operator who followed the help produced a file the loader /// refused. #[arg(long)] - prompts: Option, + prompts: Option, // ------------------------------------------------------------------ // PERF-025 — the APR-PERF-GATE-001 v2.2 §4.4 conformant mode. @@ -1795,10 +1799,10 @@ pub enum LlmSubcommand { band: bool, /// Directory receiving `receipt.rN.json` and the gzipped JSONL samples. #[arg(long)] - receipt: Option, + receipt: Option, /// Concurrency levels for --band. Arm A needs `c=1` to be present. #[arg(long, default_value = "1,4,8,16")] - bands: String, + bands: FreeText, /// Full band replicates per cell (PP-LLAMA-001 v3.0 §4.3: n >= 5). /// Each replicate writes its own independently judgeable receipt. /// @@ -1816,13 +1820,13 @@ pub enum LlmSubcommand { workload: String, /// Join key: which machine measured. Required by the receipt schema. #[arg(long, required_if_eq("band", "true"), default_value = "")] - host: String, + host: FreeText, /// Join key: which accelerator served the request. #[arg(long, required_if_eq("band", "true"), default_value = "")] - accelerator: String, + accelerator: FreeText, /// Join key: which quantization the served model uses. #[arg(long, required_if_eq("band", "true"), default_value = "")] - quantization: String, + quantization: FreeText, /// The dispatch path the SERVER took — not the hardware present. #[arg(long, default_value = "unknown", value_parser = ["cpu", "cuda", "metal", "wgpu", "unknown"])] @@ -1834,14 +1838,14 @@ pub enum LlmSubcommand { /// pointing that check at the measuring binary instead of the measured /// one would make it read green while checking nothing. #[arg(long = "server-feature")] - server_features: Vec, + server_features: Vec, /// How tokens were counted (§4.4.6). Absence of a valid value is fatal. #[arg(long, default_value = "server_usage", value_parser = ["server_usage", "client_tokenizer"])] tokenization: String, /// Tokenizer digest, required when --tokenization client_tokenizer. #[arg(long)] - tokenizer_sha256: Option, + tokenizer_sha256: Option, /// Whether the token counts include special tokens (§4.4.6). #[arg(long)] counts_special_tokens: bool, @@ -1857,7 +1861,7 @@ pub enum LlmSubcommand { /// `$(git rev-parse HEAD)`: a full 40-character lowercase object name, /// because an abbreviated sha cannot be tested for ancestry (PP-18). #[arg(long)] - commit: Option, + commit: Option, /// Who owes the comparator measurement this producer refuses to invent. /// /// §4.7.1: without `--comparator-url` every cell is NOT_APPLICABLE or @@ -1865,7 +1869,7 @@ pub enum LlmSubcommand { /// SAME run (PP-3), joined on the PP-22 key and driven by this same /// client binary (PP-25); one lane cannot produce one. #[arg(long, default_value = "perf-gate")] - comparator_owner: String, + comparator_owner: FreeText, // ------------------------------------------------------------------ // PP-LLAMA-001 v3.0 §4.3 / §5.3 — the COMPARATOR LANE. @@ -1888,25 +1892,25 @@ pub enum LlmSubcommand { /// Comparator endpoint. Measured by THIS binary, interleaved with the /// subject lane band by band, and joined on a shared `run_id`. #[arg(long)] - comparator_url: Option, + comparator_url: Option, /// Model name the comparator server expects in the request body. /// Defaults to --model; `llama-server` serves whatever it was /// launched with and ignores the field. #[arg(long)] - comparator_model: Option, + comparator_model: Option, /// PP-20: the comparator build's upstream commit. #[arg(long)] - comparator_commit: Option, + comparator_commit: Option, /// PP-20: the `cmake` line the comparator was configured with. #[arg(long)] - comparator_cmake: Option, + comparator_cmake: Option, /// PP-20: the comparator binary's sha256, 64 lowercase hex. #[arg(long)] - comparator_sha256: Option, + comparator_sha256: Option, /// PP-20: the instant after which every ratio against this pin is /// COMPARATOR_STALE, as `YYYY-MM-DDTHH:MM:SS.mmmZ`. #[arg(long)] - comparator_pin_expiry: Option, + comparator_pin_expiry: Option, // ------------------------------------------------------------------ // PP-22 / §5.3 — the comparator lane's CONFIGURATION, declared. @@ -1938,10 +1942,10 @@ pub enum LlmSubcommand { /// server resolved it to, and a guessed join-key field silently joins /// two different configurations. #[arg(long)] - comparator_fa: Option, + comparator_fa: Option, /// §5.3: `-ctk/-ctv` as launched, e.g. `f16`. #[arg(long)] - comparator_kv_type: Option, + comparator_kv_type: Option, /// PP-26: `witness.json` from /// `scripts/perf041_batched_parity_probe.py`. @@ -1952,12 +1956,12 @@ pub enum LlmSubcommand { /// emitting one token id forever at full speed, and every throughput /// number it produced was arithmetically correct and meaningless. #[arg(long)] - witness_json: Option, + witness_json: Option, /// PP-18: the `apr serve` binary that served the bands, when it is not /// this same `apr`. Hashed into `provenance.subject`. #[arg(long)] - subject_binary: Option, + subject_binary: Option, /// PP-21: sign each receipt with this key id after writing it. /// @@ -1966,11 +1970,11 @@ pub enum LlmSubcommand { /// non-zero exit is fatal: an unsigned receipt reported as signed is /// exactly the document `ArmC-sig` exists to catch. #[arg(long)] - key_id: Option, + key_id: Option, /// PP-21: keyring for --key-id. Defaults to /// `$APR_PERF_RECEIPT_KEYRING` inside the signing script. #[arg(long)] - keyring: Option, + keyring: Option, }, } diff --git a/crates/apr-cli/src/lib.rs b/crates/apr-cli/src/lib.rs index 76169e7b43..6c96d40788 100644 --- a/crates/apr-cli/src/lib.rs +++ b/crates/apr-cli/src/lib.rs @@ -31,6 +31,13 @@ )] use clap::{Parser, Subcommand, ValueEnum}; +// #3745 S1: every free-form argument of the clap tree below is declared through +// one of these role types (`batuta_common::cli_roles`), which is how `apr +// surface` reads its role. A raw `PathBuf`/`String` there is RED in the marker guard. +pub use batuta_common::cli_roles::{ + ConfigPath, DirPath, EncodeText, FreeText, InputFile, ModelPath, ModelRef, OutputPath, + PromptText, +}; use std::path::{Path, PathBuf}; // Contract assertions from YAML (pv codegen) @@ -48,10 +55,13 @@ pub mod verbosity; /// PERF-021: the accelerator refusal, shared by serve, run and chat. mod accel; + mod commands; pub mod error; mod output; pub mod pipe; +// #3745 S1: the surface the binary actually has, emitted by the binary. +pub mod surface; pub use error::CliError; @@ -69,9 +79,11 @@ pub mod model_pull { // `tests/falsify_auth_*.rs` can construct gates and middleware without // reaching into the private `commands` tree. pub mod serve_auth { + #[cfg(feature = "inference")] + pub use crate::commands::serve::auth::apply; #[cfg(feature = "inference")] pub use crate::commands::serve::auth::layer; - pub use crate::commands::serve::auth::{apply, AuthGate}; + pub use crate::commands::serve::auth::AuthGate; } // PMAT-923: e2e seam so `tests/ollama_api_serve_compat.rs` can build the REAL @@ -153,6 +165,14 @@ include!("dispatch.rs"); include!("dispatch_analysis.rs"); include!("lib_07.rs"); +/// G0.5 (#2582): the heap profiler's allocator lives HERE, next to `cli_main`, +/// so both `apr` bin targets (the root facade's `src/bin/apr.rs` and +/// `crates/apr-cli/src/main.rs`) get it. It used to exist only in apr-cli's +/// `main.rs`, one of the ways the two hand-copied entry points had diverged. +#[cfg(feature = "dhat-heap")] +#[global_allocator] +static ALLOC: dhat::Alloc = dhat::Alloc; + /// Full CLI entry point for `cargo install aprender`. /// /// This function encapsulates the complete `apr` binary logic so that @@ -162,6 +182,8 @@ pub fn cli_main() -> std::process::ExitCode { // GH-667: Reset SIGPIPE to default so piping to head/less doesn't panic. #[cfg(unix)] #[allow(unsafe_code)] + // SAFETY: signal(SIGPIPE, SIG_DFL) is async-signal-safe per POSIX. + // Called once at program start before any threads are spawned. unsafe { libc::signal(libc::SIGPIPE, libc::SIG_DFL); } @@ -169,6 +191,8 @@ pub fn cli_main() -> std::process::ExitCode { // GH-646: Clear FPCR.FZ16 on aarch64 so f16 subnormals work. #[cfg(target_arch = "aarch64")] #[allow(unsafe_code)] + // SAFETY: Reading/writing FPCR only affects floating-point behavior for the + // current thread. Called once at program start before any FP operations. unsafe { let fpcr: u64; core::arch::asm!("mrs {}, fpcr", out(reg) fpcr); @@ -178,6 +202,9 @@ pub fn cli_main() -> std::process::ExitCode { } } + #[cfg(feature = "dhat-heap")] + let _profiler = dhat::Profiler::new_heap(); + // GH-662: Respect NO_COLOR env var and non-TTY output. let no_color = std::env::var("NO_COLOR").is_ok(); let is_tty = std::io::IsTerminal::is_terminal(&std::io::stdout()); @@ -278,3 +305,64 @@ pub fn emit_version_json() { serde_json::to_string_pretty(&body).expect("build version json") ); } + +/// G0.5 (#2582): both `apr` bin targets must BE `cli_main`, not copies of it. +#[cfg(test)] +mod entry_point_is_shared { + /// What a hand-copied prologue re-introduces. Any of these in a bin target + /// means the entry point is being duplicated again instead of shared. + const PROLOGUE_TOKENS: [&str; 7] = [ + "Cli::parse", + "execute_command", + "emit_version_json", + "global_allocator", + "set_override", + "SIGPIPE", + "fpcr", + ]; + + /// The source with `//` comments removed, so a doc comment that NAMES a + /// token (as main.rs's header does) is not read as code. + fn code_of(rel: &str) -> String { + let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join(rel); + let src = + std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display())); + src.lines() + .map(|l| l.split("//").next().unwrap_or("")) + .collect::>() + .join("\n") + } + + fn violations(code: &str) -> Vec<&'static str> { + let mut v: Vec<&'static str> = PROLOGUE_TOKENS + .iter() + .copied() + .filter(|t| code.contains(t)) + .collect(); + if !code.contains("apr_cli::cli_main()") { + v.push("does not call apr_cli::cli_main()"); + } + v + } + + #[test] + fn both_apr_bins_delegate_to_cli_main() { + for rel in ["src/main.rs", "../../src/bin/apr.rs"] { + let v = violations(&code_of(rel)); + assert!(v.is_empty(), "{rel}: {v:?}"); + } + } + + /// Case table: the detector must fire on the pre-#2582 main.rs shape and + /// stay quiet on a comment that only names the tokens. + #[test] + fn detector_case_table() { + let copied = "use apr_cli::{execute_command, Cli};\nfn main() { let cli = Cli::parse(); }"; + assert!(violations(copied).contains(&"Cli::parse")); + assert!(violations(copied).contains(&"does not call apr_cli::cli_main()")); + let alloc = "static ALLOC: dhat::Alloc = dhat::Alloc; #[global_allocator]\nfn main() { apr_cli::cli_main() }"; + assert_eq!(violations(alloc), vec!["global_allocator"]); + let clean = "fn main() -> std::process::ExitCode {\n apr_cli::cli_main()\n}"; + assert!(violations(clean).is_empty()); + } +} diff --git a/crates/apr-cli/src/lib_07.rs b/crates/apr-cli/src/lib_07.rs index d5f325796b..347f39cce8 100644 --- a/crates/apr-cli/src/lib_07.rs +++ b/crates/apr-cli/src/lib_07.rs @@ -21,4 +21,5 @@ include!("lib_parse_serve.rs"); include!("lib_dispatch_coverage.rs"); include!("lib_dogfood_2392.rs"); include!("lib_falsify_2606_mcp_serve_argv.rs"); +include!("lib_parse_ptx_debug_4062.rs"); } diff --git a/crates/apr-cli/src/lib_dispatch_coverage.rs b/crates/apr-cli/src/lib_dispatch_coverage.rs index 82f47c4684..89b1f7d6ed 100644 --- a/crates/apr-cli/src/lib_dispatch_coverage.rs +++ b/crates/apr-cli/src/lib_dispatch_coverage.rs @@ -10,7 +10,7 @@ fn test_dispatch_model_commands_returns_none_for_inspect() { // Inspect is an inspection command, not a model management command let cli = make_cli(Commands::Inspect { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), vocab: false, filters: false, weights: false, @@ -32,7 +32,7 @@ #[test] fn test_dispatch_model_commands_pull_nonexistent() { let cli = make_cli(Commands::Pull { - model_ref: "nonexistent-model-that-does-not-exist-xyz123".to_string(), + model_ref: "nonexistent-model-that-does-not-exist-xyz123".to_string().into(), repo: None, force: false, dry_run: false, @@ -50,7 +50,7 @@ #[test] fn test_dispatch_model_commands_rm_nonexistent() { let cli = make_cli(Commands::Rm { - model_ref: "nonexistent-model-xyz789".to_string(), + model_ref: "nonexistent-model-xyz789".to_string().into(), }); let result = dispatch_model_commands(&cli); assert!(result.is_some(), "Rm should be handled by dispatch_model_commands"); @@ -66,7 +66,7 @@ #[test] fn test_dispatch_model_commands_tui_with_file() { let cli = make_cli(Commands::Tui { - file: Some(PathBuf::from("/tmp/nonexistent_tui_model.apr")), + file: Some(PathBuf::from("/tmp/nonexistent_tui_model.apr").into()), }); let result = dispatch_model_commands(&cli); assert!(result.is_some(), "Tui with file should be handled by dispatch_model_commands"); @@ -76,11 +76,11 @@ fn test_dispatch_model_commands_merge_nonexistent() { let cli = make_cli(Commands::Merge { files: vec![ - PathBuf::from("/tmp/nonexistent_merge_a.apr"), - PathBuf::from("/tmp/nonexistent_merge_b.apr"), + PathBuf::from("/tmp/nonexistent_merge_a.apr").into(), + PathBuf::from("/tmp/nonexistent_merge_b.apr").into(), ], - strategy: "average".to_string(), - output: Some(PathBuf::from("/tmp/merged_out.apr")), + strategy: "average".to_string().into(), + output: Some(PathBuf::from("/tmp/merged_out.apr").into()), weights: None, base_model: None, drop_rate: 0.9, @@ -99,10 +99,10 @@ fn test_dispatch_model_commands_merge_plan_mode() { let cli = make_cli(Commands::Merge { files: vec![ - PathBuf::from("/tmp/nonexistent_merge_plan_a.apr"), - PathBuf::from("/tmp/nonexistent_merge_plan_b.apr"), + PathBuf::from("/tmp/nonexistent_merge_plan_a.apr").into(), + PathBuf::from("/tmp/nonexistent_merge_plan_b.apr").into(), ], - strategy: "slerp".to_string(), + strategy: "slerp".to_string().into(), output: None, // plan mode doesn't need output weights: Some(vec![0.5, 0.5]), base_model: None, @@ -119,7 +119,7 @@ #[test] fn test_dispatch_model_commands_returns_none_for_validate() { let cli = make_cli(Commands::Validate { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), quality: false, strict: false, min_score: None, @@ -131,7 +131,7 @@ #[test] fn test_dispatch_model_commands_returns_none_for_debug() { let cli = make_cli(Commands::Debug { - file: Some(PathBuf::from("model.apr")), + file: Some(PathBuf::from("model.apr").into()), action: None, drama: false, hex: false, @@ -145,11 +145,11 @@ #[test] fn test_dispatch_model_commands_prune_nonexistent() { let cli = make_cli(Commands::ModelOps(ModelOpsCommands::Prune { - file: PathBuf::from("/tmp/nonexistent_prune_model.apr"), - method: "magnitude".to_string(), + file: PathBuf::from("/tmp/nonexistent_prune_model.apr").into(), + method: "magnitude".to_string().into(), target_ratio: 0.5, sparsity: 0.0, - output: Some(PathBuf::from("/tmp/pruned_out.apr")), + output: Some(PathBuf::from("/tmp/pruned_out.apr").into()), remove_layers: None, analyze: false, plan: false, @@ -173,7 +173,7 @@ fn test_dispatch_analysis_returns_none_for_chat() { // Chat is Extended but not an analysis command let cli = make_cli(Commands::Extended(ExtendedCommands::Chat { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), temperature: 0.7, top_p: 0.9, max_tokens: 512, @@ -196,7 +196,7 @@ #[test] fn test_dispatch_analysis_tree_nonexistent_file() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tree { - file: PathBuf::from("/tmp/nonexistent_tree_model.apr"), + file: PathBuf::from("/tmp/nonexistent_tree_model.apr").into(), filter: None, format: crate::commands::tree::TreeFormat::Ascii, sizes: false, @@ -210,7 +210,7 @@ #[test] fn test_dispatch_analysis_hex_nonexistent_file() { let cli = make_cli(Commands::Extended(ExtendedCommands::Hex { - file: PathBuf::from("/tmp/nonexistent_hex_model.apr"), + file: PathBuf::from("/tmp/nonexistent_hex_model.apr").into(), tensor: None, limit: 64, stats: false, @@ -222,7 +222,7 @@ contract: false, entropy: false, raw: false, - offset: "0".to_string(), + offset: "0".to_string().into(), width: 16, slice: None, })); @@ -234,9 +234,9 @@ #[test] fn test_dispatch_analysis_flow_nonexistent_file() { let cli = make_cli(Commands::Extended(ExtendedCommands::Flow { - file: PathBuf::from("/tmp/nonexistent_flow_model.apr"), + file: PathBuf::from("/tmp/nonexistent_flow_model.apr").into(), layer: None, - component: "full".to_string(), + component: "full".to_string().into(), verbose: false, json: false, })); @@ -248,8 +248,8 @@ #[test] fn test_dispatch_analysis_qualify_nonexistent_file() { let cli = make_cli(Commands::Extended(ExtendedCommands::Qualify { - file: PathBuf::from("/tmp/nonexistent_qualify_model.apr"), - tier: "basic".to_string(), + file: PathBuf::from("/tmp/nonexistent_qualify_model.apr").into(), + tier: "basic".to_string().into(), timeout: 30, json: false, verbose: false, @@ -263,7 +263,7 @@ #[test] fn test_dispatch_analysis_diagnose_nonexistent() { let cli = make_cli(Commands::Extended(ExtendedCommands::Diagnose { - checkpoint_dir: PathBuf::from("/tmp/nonexistent_diagnose_dir"), + checkpoint_dir: PathBuf::from("/tmp/nonexistent_diagnose_dir").into(), data: None, model_size: None, num_classes: 5, @@ -335,9 +335,9 @@ fn test_dispatch_analysis_probar_rejects_unknown_format() { let cli = make_cli(Commands::Extended(ExtendedCommands::Test { command: TestSubcommand::Tensor { - file: PathBuf::from("/tmp/nonexistent_probar_model.apr"), - output: PathBuf::from("/tmp/nonexistent_probar_out"), - format: "bogus".to_string(), + file: PathBuf::from("/tmp/nonexistent_probar_model.apr").into(), + output: PathBuf::from("/tmp/nonexistent_probar_out").into(), + format: "bogus".to_string().into(), golden: None, layer: None, assert: false, @@ -362,9 +362,9 @@ for format in ["json", "png", "both", "all"] { let cli = make_cli(Commands::Extended(ExtendedCommands::Test { command: TestSubcommand::Tensor { - file: PathBuf::from("/tmp/nonexistent_probar_model.apr"), - output: PathBuf::from("/tmp/nonexistent_probar_out"), - format: format.to_string(), + file: PathBuf::from("/tmp/nonexistent_probar_model.apr").into(), + output: PathBuf::from("/tmp/nonexistent_probar_out").into(), + format: format.to_string().into(), golden: None, layer: None, assert: false, @@ -384,8 +384,8 @@ #[test] fn test_dispatch_analysis_compare_hf_offline_rejected() { let mut cli = make_cli(Commands::Extended(ExtendedCommands::CompareHf { - file: PathBuf::from("/tmp/nonexistent_compare_hf.apr"), - hf: "org/repo".to_string(), + file: PathBuf::from("/tmp/nonexistent_compare_hf.apr").into(), + hf: "org/repo".to_string().into(), tensor: None, threshold: 0.01, json: false, @@ -409,7 +409,7 @@ #[test] fn test_dispatch_profiling_returns_none_for_chat() { let cli = make_cli(Commands::Extended(ExtendedCommands::Chat { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), temperature: 0.7, top_p: 0.9, max_tokens: 512, @@ -441,7 +441,7 @@ #[test] fn ptx_without_analyzer_reports_the_feature_not_the_path() { let cli = make_cli(Commands::Extended(ExtendedCommands::Ptx { - file: Some(PathBuf::from("/nonexistent_ptx_source_xyz.ptx")), + file: Some(PathBuf::from("/nonexistent_ptx_source_xyz.ptx").into()), kernel: None, strict: false, bugs: false, @@ -473,7 +473,7 @@ fn ptx_without_analyzer_reports_remedy_for_kernel_form_too() { let cli = make_cli(Commands::Extended(ExtendedCommands::Ptx { file: None, - kernel: Some("Q4KGemv".to_string()), + kernel: Some("Q4KGemv".to_string().into()), strict: false, bugs: false, json: false, @@ -492,7 +492,7 @@ #[test] fn test_dispatch_profiling_bench_nonexistent() { let cli = make_cli(Commands::Extended(ExtendedCommands::Bench { - file: PathBuf::from("/tmp/nonexistent_bench_model.apr"), + file: PathBuf::from("/tmp/nonexistent_bench_model.apr").into(), warmup: 1, iterations: 1, max_tokens: 10, @@ -509,9 +509,9 @@ #[test] fn test_dispatch_profiling_profile_nonexistent() { let cli = make_cli(Commands::Extended(ExtendedCommands::Profile { - file: PathBuf::from("/tmp/nonexistent_profile_model.apr"), + file: PathBuf::from("/tmp/nonexistent_profile_model.apr").into(), granular: false, - format: "text".to_string(), + format: "text".to_string().into(), focus: None, detect_naive: false, threshold: 0.01, @@ -540,8 +540,8 @@ #[test] fn test_dispatch_profiling_eval_nonexistent() { let cli = make_cli(Commands::Extended(ExtendedCommands::Eval { - file: PathBuf::from("/tmp/nonexistent_eval_model.apr"), - dataset: "test".to_string(), + file: PathBuf::from("/tmp/nonexistent_eval_model.apr").into(), + dataset: "test".to_string().into(), text: None, max_tokens: 32, threshold: 90.0, @@ -562,7 +562,7 @@ #[test] fn test_dispatch_profiling_qa_nonexistent() { let cli = make_cli(Commands::Extended(ExtendedCommands::Qa { - file: PathBuf::from("/tmp/nonexistent_qa_model.apr"), + file: PathBuf::from("/tmp/nonexistent_qa_model.apr").into(), assert_tps: None, assert_speedup: None, assert_gpu_speedup: None, @@ -599,14 +599,14 @@ // Train Plan with nonexistent data — tests train routing path let cli = make_cli(Commands::Extended(ExtendedCommands::Train { command: TrainCommands::Plan { - data: Some(PathBuf::from("/tmp/nonexistent_train_data.jsonl")), - model_size: "tiny".to_string(), + data: Some(PathBuf::from("/tmp/nonexistent_train_data.jsonl").into()), + model_size: "tiny".to_string().into(), model_path: None, num_classes: 5, - task: "classify".to_string(), + task: "classify".to_string().into(), config: None, - output: PathBuf::from("/tmp/train_plan_out"), - strategy: "auto".to_string(), + output: PathBuf::from("/tmp/train_plan_out").into(), + strategy: "auto".to_string().into(), budget: 100, scout: false, max_epochs: 3, @@ -615,7 +615,7 @@ batch_size: Some(4), val_data: None, test_data: None, - format: "text".to_string(), + format: "text".to_string().into(), }, })); let result = dispatch_analysis_commands(&cli); @@ -627,10 +627,10 @@ fn test_dispatch_train_sweep() { let cli = make_cli(Commands::Extended(ExtendedCommands::Train { command: TrainCommands::Sweep { - config: PathBuf::from("/tmp/nonexistent_sweep_config.toml"), - strategy: "grid".to_string(), + config: PathBuf::from("/tmp/nonexistent_sweep_config.toml").into(), + strategy: "grid".to_string().into(), num_configs: 5, - output_dir: PathBuf::from("/tmp/sweep_out"), + output_dir: PathBuf::from("/tmp/sweep_out").into(), seed: 42, }, })); @@ -643,7 +643,7 @@ fn test_dispatch_train_cluster_status() { let cli = make_cli(Commands::Extended(ExtendedCommands::Train { command: TrainCommands::ClusterStatus { - cluster: PathBuf::from("/tmp/nonexistent_cluster_config.yaml"), + cluster: PathBuf::from("/tmp/nonexistent_cluster_config.yaml").into(), }, })); let result = dispatch_analysis_commands(&cli); @@ -657,7 +657,7 @@ #[test] fn test_dispatch_inspection_routes_inspect() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), vocab: false, filters: false, weights: false, @@ -671,7 +671,7 @@ #[test] fn test_dispatch_inspection_routes_debug() { let cli = make_cli(Commands::Debug { - file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr")), + file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr").into()), action: None, drama: false, hex: false, @@ -685,7 +685,7 @@ #[test] fn test_dispatch_inspection_routes_validate() { let cli = make_cli(Commands::Validate { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), strict: false, quality: false, min_score: None, @@ -697,7 +697,7 @@ #[test] fn test_dispatch_inspection_routes_lint() { let cli = make_cli(Commands::Lint { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), strict: false, }); let result = dispatch_inspection_commands(&cli); @@ -707,8 +707,8 @@ #[test] fn test_dispatch_inspection_returns_none_for_export() { let cli = make_cli(Commands::Export { - file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr")), - format: "gguf".to_string(), + file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr").into()), + format: "gguf".to_string().into(), output: None, quantize: None, list_formats: false, @@ -728,7 +728,7 @@ #[test] fn test_dispatch_diagnostic_routes_trace() { let cli = make_cli(Commands::Trace { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), layer: None, reference: None, json: false, @@ -738,7 +738,7 @@ interactive: false, save_tensor: None, save_tensor_dir: None, - save_tensor_layers: "0..1".to_string(), + save_tensor_layers: "0..1".to_string().into(), }); let result = dispatch_diagnostic_commands(&cli); assert!(result.is_some(), "Trace should be handled by diagnostic dispatcher"); @@ -747,7 +747,7 @@ #[test] fn test_dispatch_diagnostic_routes_tensors() { let cli = make_cli(Commands::Tensors { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), stats: false, filter: None, limit: 0, @@ -760,8 +760,8 @@ #[test] fn test_dispatch_diagnostic_routes_diff() { let cli = make_cli(Commands::Diff { - file1: PathBuf::from("/tmp/nonexistent_a.apr"), - file2: PathBuf::from("/tmp/nonexistent_b.apr"), + file1: PathBuf::from("/tmp/nonexistent_a.apr").into(), + file2: PathBuf::from("/tmp/nonexistent_b.apr").into(), weights: false, values: false, filter: None, @@ -779,7 +779,7 @@ #[test] fn test_dispatch_diagnostic_returns_none_for_inspect() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), vocab: false, filters: false, weights: false, @@ -797,9 +797,9 @@ #[test] fn test_dispatch_format_routes_import() { let cli = make_cli(Commands::Import { - source: "/tmp/nonexistent_pmat540.safetensors".to_string(), + source: "/tmp/nonexistent_pmat540.safetensors".to_string().into(), output: None, - arch: "auto".to_string(), + arch: "auto".to_string().into(), quantize: None, strict: false, preserve_q4k: false, @@ -814,10 +814,10 @@ #[test] fn test_dispatch_format_routes_convert() { let cli = make_cli(Commands::Convert { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), quantize: None, compress: None, - output: PathBuf::from("/tmp/nonexistent_pmat540_out.apr"), + output: PathBuf::from("/tmp/nonexistent_pmat540_out.apr").into(), force: false, }); let result = dispatch_format_commands(&cli); @@ -827,8 +827,8 @@ #[test] fn test_dispatch_format_routes_export() { let cli = make_cli(Commands::Export { - file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr")), - format: "gguf".to_string(), + file: Some(PathBuf::from("/tmp/nonexistent_pmat540.apr").into()), + format: "gguf".to_string().into(), output: None, quantize: None, list_formats: false, @@ -844,7 +844,7 @@ #[test] fn test_dispatch_format_returns_none_for_inspect() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), vocab: false, filters: false, weights: false, @@ -862,7 +862,7 @@ #[test] fn test_dispatch_runtime_routes_check() { let cli = make_cli(Commands::Check { - file: PathBuf::from("/tmp/nonexistent_pmat540.gguf"), + file: PathBuf::from("/tmp/nonexistent_pmat540.gguf").into(), no_gpu: true, json: false, }); @@ -873,7 +873,7 @@ #[test] fn test_dispatch_runtime_returns_none_for_inspect() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), vocab: false, filters: false, weights: false, @@ -891,7 +891,7 @@ #[test] fn test_dispatch_core_routes_inspect_via_inspection() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), vocab: false, filters: false, weights: false, @@ -906,7 +906,7 @@ #[test] fn test_dispatch_core_routes_tensors_via_diagnostic() { let cli = make_cli(Commands::Tensors { - file: PathBuf::from("/tmp/nonexistent_pmat540.apr"), + file: PathBuf::from("/tmp/nonexistent_pmat540.apr").into(), stats: false, filter: None, limit: 0, @@ -919,9 +919,9 @@ #[test] fn test_dispatch_core_routes_import_via_format() { let cli = make_cli(Commands::Import { - source: "/tmp/nonexistent_pmat540.safetensors".to_string(), + source: "/tmp/nonexistent_pmat540.safetensors".to_string().into(), output: None, - arch: "auto".to_string(), + arch: "auto".to_string().into(), quantize: None, strict: false, preserve_q4k: false, @@ -936,7 +936,7 @@ #[test] fn test_dispatch_core_returns_none_for_extended() { let cli = make_cli(Commands::Extended(ExtendedCommands::Chat { - file: PathBuf::from("/tmp/nonexistent_pmat540.gguf"), + file: PathBuf::from("/tmp/nonexistent_pmat540.gguf").into(), temperature: 0.7, top_p: 0.9, max_tokens: 512, diff --git a/crates/apr-cli/src/lib_execute_export_convert.rs b/crates/apr-cli/src/lib_execute_export_convert.rs index 6ecab9f4d2..6713274f75 100644 --- a/crates/apr-cli/src/lib_execute_export_convert.rs +++ b/crates/apr-cli/src/lib_execute_export_convert.rs @@ -3,9 +3,9 @@ #[test] fn test_execute_export_file_not_found() { let cli = make_cli(Commands::Export { - file: Some(PathBuf::from("/tmp/nonexistent_model_export_test.apr")), - format: "safetensors".to_string(), - output: Some(PathBuf::from("/tmp/out.safetensors")), + file: Some(PathBuf::from("/tmp/nonexistent_model_export_test.apr").into()), + format: "safetensors".to_string().into(), + output: Some(PathBuf::from("/tmp/out.safetensors").into()), quantize: None, list_formats: false, batch: None, @@ -21,10 +21,10 @@ #[test] fn test_execute_convert_file_not_found() { let cli = make_cli(Commands::Convert { - file: PathBuf::from("/tmp/nonexistent_model_convert_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_convert_test.apr").into(), quantize: None, compress: None, - output: PathBuf::from("/tmp/out.apr"), + output: PathBuf::from("/tmp/out.apr").into(), force: false, }); let result = execute_command(&cli); @@ -38,7 +38,7 @@ #[test] fn test_execute_hex_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Hex { - file: PathBuf::from("/tmp/nonexistent_model_hex_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_hex_test.apr").into(), tensor: None, limit: 64, stats: false, @@ -50,7 +50,7 @@ contract: false, entropy: false, raw: false, - offset: String::new(), + offset: String::new().into(), width: 16, slice: None, })); @@ -62,7 +62,7 @@ #[test] fn test_execute_tree_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tree { - file: PathBuf::from("/tmp/nonexistent_model_tree_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_tree_test.apr").into(), filter: None, format: crate::commands::tree::TreeFormat::Ascii, sizes: false, @@ -76,9 +76,9 @@ #[test] fn test_execute_flow_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Flow { - file: PathBuf::from("/tmp/nonexistent_model_flow_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_flow_test.apr").into(), layer: None, - component: "full".to_string(), + component: "full".to_string().into(), verbose: false, json: false, })); @@ -93,9 +93,9 @@ use TestSubcommand; let cli = make_cli(Commands::Extended(ExtendedCommands::Test { command: TestSubcommand::Tensor { - file: PathBuf::from("/tmp/nonexistent_model_probar_test.apr"), - output: PathBuf::from("/tmp/probar-out"), - format: "both".to_string(), + file: PathBuf::from("/tmp/nonexistent_model_probar_test.apr").into(), + output: PathBuf::from("/tmp/probar-out").into(), + format: "both".to_string().into(), golden: None, layer: None, assert: false, @@ -110,7 +110,7 @@ #[test] fn test_execute_check_file_not_found() { let cli = make_cli(Commands::Check { - file: PathBuf::from("/tmp/nonexistent_model_check_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_check_test.apr").into(), no_gpu: true, json: false, }); @@ -148,7 +148,7 @@ #[test] fn test_execute_explain_with_code() { let cli = make_cli(Commands::Explain { - code_or_file: Some("E001".to_string()), + code_or_file: Some("E001".to_string().into()), file: None, tensor: None, kernel: false, @@ -166,18 +166,18 @@ fn test_execute_tune_plan_no_file() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tune { file: None, - method: "auto".to_string(), + method: "auto".to_string().into(), rank: None, vram: 16.0, plan: true, - model: Some("7B".to_string()), + model: Some("7B".to_string().into()), freeze_base: false, train_data: None, json: false, task: None, budget: 10, - strategy: "tpe".to_string(), - scheduler: "asha".to_string(), + strategy: "tpe".to_string().into(), + scheduler: "asha".to_string().into(), scout: false, data: None, num_classes: 5, @@ -200,7 +200,7 @@ #[test] fn test_execute_qa_all_skips_succeeds() { let cli = make_cli(Commands::Extended(ExtendedCommands::Qa { - file: PathBuf::from("/tmp/nonexistent_model_qa_test.gguf"), + file: PathBuf::from("/tmp/nonexistent_model_qa_test.gguf").into(), assert_tps: None, assert_speedup: None, assert_gpu_speedup: None, @@ -236,7 +236,7 @@ #[test] fn test_execute_qa_with_gates_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Qa { - file: PathBuf::from("/tmp/nonexistent_model_qa_gates_test.gguf"), + file: PathBuf::from("/tmp/nonexistent_model_qa_gates_test.gguf").into(), assert_tps: None, assert_speedup: None, assert_gpu_speedup: None, @@ -272,9 +272,9 @@ #[test] fn test_execute_import_invalid_source() { let cli = make_cli(Commands::Import { - source: "/tmp/nonexistent_model_import_test.gguf".to_string(), + source: "/tmp/nonexistent_model_import_test.gguf".to_string().into(), output: None, - arch: "auto".to_string(), + arch: "auto".to_string().into(), quantize: None, strict: false, preserve_q4k: false, diff --git a/crates/apr-cli/src/lib_extract_paths.rs b/crates/apr-cli/src/lib_extract_paths.rs index 51e20478eb..41978db33b 100644 --- a/crates/apr-cli/src/lib_extract_paths.rs +++ b/crates/apr-cli/src/lib_extract_paths.rs @@ -3,7 +3,7 @@ #[test] fn test_extract_paths_tree_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tree { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), filter: None, format: crate::commands::tree::TreeFormat::Ascii, sizes: false, @@ -17,9 +17,9 @@ #[test] fn test_extract_paths_flow_exempt() { let cmd = Commands::Extended(ExtendedCommands::Flow { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), layer: None, - component: "full".to_string(), + component: "full".to_string().into(), verbose: false, json: false, }); @@ -31,11 +31,11 @@ #[test] fn test_extract_paths_publish_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Publish { - directory: PathBuf::from("/tmp/models"), - repo_id: "org/repo".to_string(), + directory: PathBuf::from("/tmp/models").into(), + repo_id: "org/repo".to_string().into(), model_name: None, - license: "mit".to_string(), - pipeline_tag: "text-generation".to_string(), + license: "mit".to_string().into(), + pipeline_tag: "text-generation".to_string().into(), library_name: None, tags: None, message: None, @@ -50,8 +50,8 @@ #[test] fn test_extract_paths_tune_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tune { - file: Some(PathBuf::from("model.apr")), - method: "auto".to_string(), + file: Some(PathBuf::from("model.apr").into()), + method: "auto".to_string().into(), rank: None, vram: 16.0, plan: false, @@ -61,8 +61,8 @@ json: false, task: None, budget: 10, - strategy: "tpe".to_string(), - scheduler: "asha".to_string(), + strategy: "tpe".to_string().into(), + scheduler: "asha".to_string().into(), scout: false, data: None, num_classes: 5, @@ -79,7 +79,7 @@ #[test] fn test_extract_paths_pull_exempt() { let cmd = Commands::Pull { - model_ref: "hf://org/repo".to_string(), + model_ref: "hf://org/repo".to_string().into(), repo: None, force: false, dry_run: false, @@ -97,7 +97,7 @@ #[test] fn test_extract_paths_rm_exempt() { let cmd = Commands::Rm { - model_ref: "model-name".to_string(), + model_ref: "model-name".to_string().into(), }; let paths = extract_model_paths(&cmd); assert!(paths.is_empty(), "Rm is a diagnostic command (exempt)"); @@ -108,8 +108,8 @@ fn test_extract_paths_canary_exempt() { let cmd = Commands::Canary { command: CanaryCommands::Check { - file: PathBuf::from("model.apr"), - canary: PathBuf::from("canary.json"), + file: PathBuf::from("model.apr").into(), + canary: PathBuf::from("canary.json").into(), }, }; let paths = extract_model_paths(&cmd); @@ -120,7 +120,7 @@ #[test] fn test_extract_paths_oracle_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Oracle { - source: Some("model.gguf".to_string()), + source: Some("model.gguf".to_string().into()), family: None, size: None, compliance: false, @@ -141,9 +141,9 @@ let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Showcase { auto_verify: false, step: None, - tier: "small".to_string(), - model_dir: PathBuf::from("./models"), - baseline: "llama-cpp,ollama".to_string(), + tier: "small".to_string().into(), + model_dir: PathBuf::from("./models").into(), + baseline: "llama-cpp,ollama".to_string().into(), zram: false, runs: 30, gpu: false, @@ -163,8 +163,8 @@ fn test_extract_paths_rosetta_convert() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Convert { - source: PathBuf::from("model.gguf"), - target: PathBuf::from("out.safetensors"), + source: PathBuf::from("model.gguf").into(), + target: PathBuf::from("out.safetensors").into(), quantize: None, verify: false, json: false, @@ -180,9 +180,9 @@ fn test_extract_paths_rosetta_chain() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Chain { - source: PathBuf::from("model.gguf"), - formats: vec!["safetensors".to_string(), "apr".to_string()], - work_dir: PathBuf::from("/tmp"), + source: PathBuf::from("model.gguf").into(), + formats: vec!["safetensors".to_string().into(), "apr".to_string().into()], + work_dir: PathBuf::from("/tmp").into(), json: false, }, })); @@ -195,8 +195,8 @@ fn test_extract_paths_rosetta_verify() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Verify { - source: PathBuf::from("model.apr"), - intermediate: "safetensors".to_string(), + source: PathBuf::from("model.apr").into(), + intermediate: "safetensors".to_string().into(), tolerance: 1e-5, json: false, }, @@ -210,9 +210,9 @@ fn test_extract_paths_rosetta_compare_inference() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::CompareInference { - model_a: PathBuf::from("model_a.gguf"), - model_b: PathBuf::from("model_b.apr"), - prompt: "test".to_string(), + model_a: PathBuf::from("model_a.gguf").into(), + model_b: PathBuf::from("model_b.apr").into(), + prompt: "test".to_string().into(), max_tokens: 5, temperature: 0.0, tolerance: 0.1, @@ -231,7 +231,7 @@ fn test_extract_paths_rosetta_inspect_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Inspect { - file: PathBuf::from("model.gguf"), + file: PathBuf::from("model.gguf").into(), hexdump: false, json: false, }, @@ -248,8 +248,8 @@ fn test_extract_paths_rosetta_diff_tensors_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::DiffTensors { - model_a: PathBuf::from("a.gguf"), - model_b: PathBuf::from("b.apr"), + model_a: PathBuf::from("a.gguf").into(), + model_b: PathBuf::from("b.apr").into(), mismatches_only: false, show_values: 0, filter: None, @@ -268,7 +268,7 @@ fn test_extract_paths_rosetta_fingerprint_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Fingerprint { - model: PathBuf::from("model.gguf"), + model: PathBuf::from("model.gguf").into(), model_b: None, output: None, filter: None, @@ -288,7 +288,7 @@ fn test_extract_paths_rosetta_validate_stats_exempt() { let cmd = Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::ValidateStats { - model: PathBuf::from("model.apr"), + model: PathBuf::from("model.apr").into(), reference: None, fingerprints: None, threshold: 3.0, @@ -353,7 +353,7 @@ #[test] fn test_execute_inspect_file_not_found() { let cli = make_cli(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_model_inspect_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_inspect_test.apr").into(), vocab: false, filters: false, weights: false, @@ -371,7 +371,7 @@ #[test] fn test_execute_debug_file_not_found() { let cli = make_cli(Commands::Debug { - file: Some(PathBuf::from("/tmp/nonexistent_model_debug_test.apr")), + file: Some(PathBuf::from("/tmp/nonexistent_model_debug_test.apr").into()), action: None, drama: false, hex: false, @@ -386,7 +386,7 @@ #[test] fn test_execute_validate_file_not_found() { let cli = make_cli(Commands::Validate { - file: PathBuf::from("/tmp/nonexistent_model_validate_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_validate_test.apr").into(), quality: false, strict: false, min_score: None, @@ -402,8 +402,8 @@ #[test] fn test_execute_diff_file_not_found() { let cli = make_cli(Commands::Diff { - file1: PathBuf::from("/tmp/nonexistent_model_diff1.apr"), - file2: PathBuf::from("/tmp/nonexistent_model_diff2.apr"), + file1: PathBuf::from("/tmp/nonexistent_model_diff1.apr").into(), + file2: PathBuf::from("/tmp/nonexistent_model_diff2.apr").into(), weights: false, values: false, filter: None, @@ -422,7 +422,7 @@ #[test] fn test_execute_tensors_file_not_found() { let cli = make_cli(Commands::Tensors { - file: PathBuf::from("/tmp/nonexistent_model_tensors_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_tensors_test.apr").into(), stats: false, filter: None, limit: 0, @@ -439,7 +439,7 @@ #[test] fn test_execute_lint_file_not_found() { let cli = make_cli(Commands::Lint { - file: PathBuf::from("/tmp/nonexistent_model_lint_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_lint_test.apr").into(), strict: false, }); let result = execute_command(&cli); @@ -450,7 +450,7 @@ #[test] fn test_execute_trace_file_not_found() { let cli = make_cli(Commands::Trace { - file: PathBuf::from("/tmp/nonexistent_model_trace_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_trace_test.apr").into(), layer: None, reference: None, json: false, @@ -460,7 +460,7 @@ interactive: false, save_tensor: None, save_tensor_dir: None, - save_tensor_layers: "0..1".to_string(), + save_tensor_layers: "0..1".to_string().into(), }); let result = execute_command(&cli); assert!(result.is_err(), "Trace should fail with non-existent file"); diff --git a/crates/apr-cli/src/lib_parse.rs b/crates/apr-cli/src/lib_parse.rs index cf733773cc..7a1cbe1a50 100644 --- a/crates/apr-cli/src/lib_parse.rs +++ b/crates/apr-cli/src/lib_parse.rs @@ -21,9 +21,9 @@ task, .. } => { - assert_eq!(input, Some(PathBuf::from("audio.wav"))); - assert_eq!(language, Some("en".to_string())); - assert_eq!(task, Some("transcribe".to_string())); + assert_eq!(input, Some(PathBuf::from("audio.wav")).map(Into::into)); + assert_eq!(language, Some("en".to_string()).map(Into::into)); + assert_eq!(task, Some("transcribe".to_string()).map(Into::into)); } _ => panic!("Expected Run command"), } @@ -89,7 +89,7 @@ assert!(skip_gpu_speedup); assert!(skip_contract); assert!(skip_format_parity); - assert_eq!(safetensors_path, Some(PathBuf::from("model.safetensors"))); + assert_eq!(safetensors_path, Some(PathBuf::from("model.safetensors")).map(Into::into)); assert_eq!(iterations, 20); assert_eq!(warmup, 5); assert_eq!(max_tokens, 64); @@ -185,9 +185,9 @@ assert_eq!(warmup, 10); assert_eq!(iterations, 20); assert_eq!(max_tokens, 64); - assert_eq!(prompt, Some("The quick brown fox".to_string())); + assert_eq!(prompt, Some("The quick brown fox".to_string()).map(Into::into)); assert!(fast); - assert_eq!(brick, Some("attention".to_string())); + assert_eq!(brick, Some("attention".to_string()).map(Into::into)); } _ => panic!("Expected Bench command"), } @@ -221,10 +221,10 @@ simulated, .. }) => { - assert_eq!(model_path, Some(PathBuf::from("model.gguf"))); + assert_eq!(model_path, Some(PathBuf::from("model.gguf")).map(Into::into)); assert!(speculative); assert_eq!(speculation_k, 8); - assert_eq!(draft_model, Some(PathBuf::from("draft.gguf"))); + assert_eq!(draft_model, Some(PathBuf::from("draft.gguf")).map(Into::into)); assert_eq!(concurrent, 4); assert!(simulated); } @@ -260,8 +260,8 @@ assert!(energy); assert!(perf_grade); assert!(callgraph); - assert_eq!(compare_hf, Some("openai/whisper-tiny".to_string())); - assert_eq!(output, Some(PathBuf::from("/tmp/flame.svg"))); + assert_eq!(compare_hf, Some("openai/whisper-tiny".to_string()).map(Into::into)); + assert_eq!(output, Some(PathBuf::from("/tmp/flame.svg")).map(Into::into)); } _ => panic!("Expected Profile command"), } @@ -300,15 +300,15 @@ profile, .. }) => { - assert_eq!(system, Some("You are a helpful assistant.".to_string())); + assert_eq!(system, Some("You are a helpful assistant.".to_string()).map(Into::into)); assert!(inspect); assert!(trace); assert_eq!( trace_steps, - Some(vec!["Tokenize".to_string(), "Decode".to_string()]) + Some(vec!["Tokenize".to_string(), "Decode".to_string()]).map(|v| v.into_iter().map(Into::into).collect()) ); assert!(trace_verbose); - assert_eq!(trace_output, Some(PathBuf::from("/tmp/chat-trace.json"))); + assert_eq!(trace_output, Some(PathBuf::from("/tmp/chat-trace.json")).map(Into::into)); assert_eq!(trace_level, "payload"); assert!(profile); } @@ -335,7 +335,7 @@ quiet, .. })) => { - assert_eq!(step, Some("bench".to_string())); + assert_eq!(step, Some("bench".to_string()).map(Into::into)); assert_eq!(tier, "tiny"); assert!(zram); assert_eq!(runs, 50); @@ -423,7 +423,7 @@ assert_eq!(model_b, PathBuf::from("test.apr")); assert!(mismatches_only); assert_eq!(show_values, 5); - assert_eq!(filter, Some("lm_head".to_string())); + assert_eq!(filter, Some("lm_head".to_string()).map(Into::into)); assert!(json); } _ => panic!("Expected DiffTensors subcommand"), diff --git a/crates/apr-cli/src/lib_parse_eval.rs b/crates/apr-cli/src/lib_parse_eval.rs index 3004151ce9..2d90e93b64 100644 --- a/crates/apr-cli/src/lib_parse_eval.rs +++ b/crates/apr-cli/src/lib_parse_eval.rs @@ -32,7 +32,7 @@ }) => { assert_eq!(file, PathBuf::from("model.gguf")); assert_eq!(dataset, "lambada"); - assert_eq!(text, Some("The quick brown fox".to_string())); + assert_eq!(text, Some("The quick brown fox".to_string()).map(Into::into)); assert_eq!(max_tokens, 256); assert!((threshold - 15.5).abs() < f32::EPSILON); assert!(task.is_none()); @@ -90,7 +90,7 @@ json: _, }) => { assert_eq!(file, PathBuf::from("model.apr")); - assert_eq!(layer, Some("encoder.0".to_string())); + assert_eq!(layer, Some("encoder.0".to_string()).map(Into::into)); assert_eq!(component, "encoder"); assert!(verbose); } @@ -145,7 +145,7 @@ .. }) => { assert_eq!(file, PathBuf::from("model.apr")); - assert_eq!(tensor, Some("embed.weight".to_string())); + assert_eq!(tensor, Some("embed.weight".to_string()).map(Into::into)); assert_eq!(limit, 128); assert!(stats); assert!(list); @@ -204,7 +204,7 @@ depth, }) => { assert_eq!(file, PathBuf::from("model.apr")); - assert_eq!(filter, Some("encoder".to_string())); + assert_eq!(filter, Some("encoder".to_string()).map(Into::into)); assert_eq!(format, crate::commands::tree::TreeFormat::Mermaid); assert!(sizes); assert_eq!(depth, Some(3)); @@ -273,8 +273,8 @@ assert_eq!(file, PathBuf::from("model.apr")); assert_eq!(output, PathBuf::from("/tmp/probar")); assert_eq!(format, "json"); - assert_eq!(golden, Some(PathBuf::from("/refs/golden"))); - assert_eq!(layer, Some("layer.0".to_string())); + assert_eq!(golden, Some(PathBuf::from("/refs/golden")).map(Into::into)); + assert_eq!(layer, Some("layer.0".to_string()).map(Into::into)); assert!(!assert); assert!((tolerance - 0.98).abs() < 0.01); }, @@ -333,7 +333,7 @@ strings, limit, } => { - assert_eq!(file, Some(PathBuf::from("model.apr"))); + assert_eq!(file, Some(PathBuf::from("model.apr")).map(Into::into)); assert!(action.is_none(), "no subcommand was given"); assert!(drama); assert!(hex); @@ -373,7 +373,7 @@ let cli = parse_cli(args).expect("Failed to parse"); match *cli.command { Commands::Tui { file } => { - assert_eq!(file, Some(PathBuf::from("model.apr"))); + assert_eq!(file, Some(PathBuf::from("model.apr")).map(Into::into)); } _ => panic!("Expected Tui command"), } @@ -424,12 +424,12 @@ allow_no_config, } => { assert_eq!(source, "hf://openai/whisper-tiny"); - assert_eq!(output, Some(PathBuf::from("whisper.apr"))); + assert_eq!(output, Some(PathBuf::from("whisper.apr")).map(Into::into)); assert_eq!(arch, "whisper"); - assert_eq!(quantize, Some("int8".to_string())); + assert_eq!(quantize, Some("int8".to_string()).map(Into::into)); assert!(strict); assert!(preserve_q4k); - assert_eq!(tokenizer, Some(PathBuf::from("/path/to/tokenizer.json"))); + assert_eq!(tokenizer, Some(PathBuf::from("/path/to/tokenizer.json")).map(Into::into)); assert!(!enforce_provenance); assert!(!allow_no_config); } diff --git a/crates/apr-cli/src/lib_parse_export.rs b/crates/apr-cli/src/lib_parse_export.rs index 2ea477e5a1..353fdcf74d 100644 --- a/crates/apr-cli/src/lib_parse_export.rs +++ b/crates/apr-cli/src/lib_parse_export.rs @@ -22,10 +22,10 @@ quantize, .. } => { - assert_eq!(file, Some(PathBuf::from("model.apr"))); + assert_eq!(file, Some(PathBuf::from("model.apr")).map(Into::into)); assert_eq!(format, "gguf"); - assert_eq!(output, Some(PathBuf::from("model.gguf"))); - assert_eq!(quantize, Some("int4".to_string())); + assert_eq!(output, Some(PathBuf::from("model.gguf")).map(Into::into)); + assert_eq!(quantize, Some("int4".to_string()).map(Into::into)); } _ => panic!("Expected Export command"), } @@ -72,8 +72,8 @@ force, } => { assert_eq!(file, PathBuf::from("model.apr")); - assert_eq!(quantize, Some("q4k".to_string())); - assert_eq!(compress, Some("zstd".to_string())); + assert_eq!(quantize, Some("q4k".to_string()).map(Into::into)); + assert_eq!(compress, Some("zstd".to_string()).map(Into::into)); assert_eq!(output, PathBuf::from("model-q4k.apr")); assert!(force); } @@ -130,7 +130,7 @@ family, size, })) => { - assert_eq!(source, Some("model.gguf".to_string())); + assert_eq!(source, Some("model.gguf".to_string()).map(Into::into)); assert!(compliance); assert!(tensors); assert!(stats); @@ -158,8 +158,8 @@ .. })) => { assert!(source.is_none()); - assert_eq!(family, Some("qwen2".to_string())); - assert_eq!(size, Some("7b".to_string())); + assert_eq!(family, Some("qwen2".to_string()).map(Into::into)); + assert_eq!(size, Some("7b".to_string()).map(Into::into)); } _ => panic!("Expected Oracle command"), } @@ -172,7 +172,7 @@ let cli = parse_cli(args).expect("Failed to parse"); match *cli.command { Commands::Extended(ExtendedCommands::Tools(ToolCommands::Oracle { source, .. })) => { - assert_eq!(source, Some("hf://Qwen/Qwen2.5-Coder-1.5B".to_string())); + assert_eq!(source, Some("hf://Qwen/Qwen2.5-Coder-1.5B".to_string()).map(Into::into)); } _ => panic!("Expected Oracle command"), } @@ -259,7 +259,7 @@ }) => { assert_eq!(file, PathBuf::from("model.apr")); assert_eq!(hf, "openai/whisper-tiny"); - assert_eq!(tensor, Some("encoder.0".to_string())); + assert_eq!(tensor, Some("encoder.0".to_string()).map(Into::into)); assert!((threshold - 1e-3).abs() < f64::EPSILON); assert!(json); } @@ -403,14 +403,14 @@ json, .. }) => { - assert_eq!(file, Some(PathBuf::from("model.apr"))); + assert_eq!(file, Some(PathBuf::from("model.apr")).map(Into::into)); assert_eq!(method, "lora"); assert_eq!(rank, Some(16)); assert!((vram - 24.0).abs() < f64::EPSILON); assert!(plan); - assert_eq!(model, Some("7B".to_string())); + assert_eq!(model, Some("7B".to_string()).map(Into::into)); assert!(freeze_base); - assert_eq!(train_data, Some(PathBuf::from("data.jsonl"))); + assert_eq!(train_data, Some(PathBuf::from("data.jsonl")).map(Into::into)); assert!(json); } _ => panic!("Expected Tune command"), diff --git a/crates/apr-cli/src/lib_parse_ptx_debug_4062.rs b/crates/apr-cli/src/lib_parse_ptx_debug_4062.rs new file mode 100644 index 0000000000..b9542ec101 --- /dev/null +++ b/crates/apr-cli/src/lib_parse_ptx_debug_4062.rs @@ -0,0 +1,64 @@ +// #4062: `apr ptx-debug` absorbs the `aprender-ptx-debug` binary. These pin +// that apr parses its argv, dispatches it to the same library code, and never +// turns a failing analysis verdict into exit 0. + +const PTX_HEADER_ONLY: &str = ".version 8.0\n.target sm_70\n.address_size 64\n"; + +#[test] +fn apr_ptx_debug_parses_the_standalone_analyze_argv() { + let cli = parse_cli(vec![ + "apr", + "ptx-debug", + "analyze", + "k.ptx", + "--min-score", + "80", + "--json", + ]) + .expect("apr ptx-debug analyze must parse"); + match *cli.command { + Commands::PtxDebug(trueno_ptx_debug::cli::Command::Analyze(a)) => { + assert_eq!(a.file, "k.ptx"); + assert!((a.min_score - 80.0).abs() < f64::EPSILON); + assert!(a.json); + } + other => panic!("expected ptx-debug analyze, got {other:?}"), + } + assert!(parse_cli(vec!["apr", "ptx-debug", "bogus"]).is_err()); +} + +#[test] +fn apr_ptx_debug_gen_fkr_dispatches_to_the_library() { + let dir = tempfile::tempdir().expect("tempdir"); + let ptx = dir.path().join("k.ptx"); + let out = dir.path().join("fkr.rs"); + std::fs::write(&ptx, PTX_HEADER_ONLY).expect("write ptx"); + let cli = parse_cli_owned(vec![ + "apr".into(), + "ptx-debug".into(), + "gen-fkr".into(), + ptx.display().to_string(), + "-o".into(), + out.display().to_string(), + ]) + .expect("apr ptx-debug gen-fkr must parse"); + let result = dispatch_sibling_cli_commands(&cli).expect("ptx-debug is a sibling CLI route"); + assert!(result.is_ok(), "gen-fkr failed: {result:?}"); + let written = std::fs::read_to_string(&out).expect("gen-fkr must write -o"); + assert!(!written.trim().is_empty()); +} + +#[test] +fn apr_ptx_debug_a_failing_verdict_is_never_exit_zero() { + assert!(ptx_debug_result(Ok(0)).is_ok()); + let mut messages = Vec::new(); + for code in [1, 2, 3, 42] { + let err = ptx_debug_result(Ok(code)).expect_err("non-zero verdict must fail"); + assert_eq!(err.exit_code_value(), 5, "verdict {code}"); + messages.push(err.to_string()); + } + messages.dedup(); + assert_eq!(messages.len(), 4, "each verdict names itself: {messages:?}"); + let io = ptx_debug_result(Err("Failed to read x.ptx".into())).expect_err("io error"); + assert!(io.to_string().contains("Failed to read x.ptx")); +} diff --git a/crates/apr-cli/src/lib_parse_rosetta.rs b/crates/apr-cli/src/lib_parse_rosetta.rs index a17aba69c8..46e4a38285 100644 --- a/crates/apr-cli/src/lib_parse_rosetta.rs +++ b/crates/apr-cli/src/lib_parse_rosetta.rs @@ -136,7 +136,7 @@ // Diagnostic commands should return no paths (exempt from validation) let diagnostic_commands = vec![ Commands::Inspect { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), vocab: false, filters: false, weights: false, @@ -144,7 +144,7 @@ quality: false, }, Commands::Debug { - file: Some(PathBuf::from("m.apr")), + file: Some(PathBuf::from("m.apr").into()), action: None, drama: false, hex: false, @@ -152,24 +152,24 @@ limit: 256, }, Commands::Validate { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), quality: false, strict: false, min_score: None, }, Commands::Tensors { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), stats: false, filter: None, limit: 0, json: false, }, Commands::Lint { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), strict: false, }, Commands::Extended(ExtendedCommands::Qa { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), assert_tps: None, assert_speedup: None, assert_gpu_speedup: None, @@ -195,7 +195,7 @@ assert_classifier_head: false, }), Commands::Extended(ExtendedCommands::Hex { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), tensor: None, limit: 64, stats: false, @@ -207,21 +207,21 @@ contract: false, entropy: false, raw: false, - offset: "0".to_string(), + offset: "0".to_string().into(), width: 16, slice: None, }), Commands::Extended(ExtendedCommands::Tree { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), filter: None, format: crate::commands::tree::TreeFormat::Ascii, sizes: false, depth: None, }), Commands::Extended(ExtendedCommands::Flow { - file: PathBuf::from("m.apr"), + file: PathBuf::from("m.apr").into(), layer: None, - component: "full".to_string(), + component: "full".to_string().into(), verbose: false, json: false, }), @@ -250,9 +250,9 @@ fn test_extract_paths_action_commands() { let serve_cmd = Commands::Serve { command: ServeCommands::Run { - file: Some(PathBuf::from("model.gguf")), + file: Some(PathBuf::from("model.gguf").into()), port: 8080, - host: "127.0.0.1".to_string(), + host: "127.0.0.1".to_string().into(), no_cors: false, no_metrics: false, no_gpu: false, @@ -261,7 +261,7 @@ list_devices: false, batch: false, trace: false, - trace_level: "basic".to_string(), + trace_level: "basic".to_string().into(), profile: false, backend: BackendArg::default(), otlp_endpoint: None, @@ -275,7 +275,7 @@ assert_eq!(paths, Vec::::new()); let bench_cmd = Commands::Extended(ExtendedCommands::Bench { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), warmup: 3, iterations: 5, max_tokens: 32, @@ -292,7 +292,7 @@ #[test] fn test_extract_paths_run_hf_url() { let cmd = Commands::Run { - source: "hf://org/repo".to_string(), + source: "hf://org/repo".to_string().into(), positional_prompt: None, input: None, prompt: None, @@ -337,12 +337,12 @@ fn test_extract_paths_merge_multiple() { let cmd = Commands::Merge { files: vec![ - PathBuf::from("a.apr"), - PathBuf::from("b.apr"), - PathBuf::from("c.apr"), + PathBuf::from("a.apr").into(), + PathBuf::from("b.apr").into(), + PathBuf::from("c.apr").into(), ], - strategy: "average".to_string(), - output: Some(PathBuf::from("merged.apr")), + strategy: "average".to_string().into(), + output: Some(PathBuf::from("merged.apr").into()), weights: None, base_model: None, drop_rate: 0.9, @@ -412,19 +412,19 @@ })) => { assert_eq!(directory, PathBuf::from("/tmp/models")); assert_eq!(repo_id, "paiml/whisper-apr-tiny"); - assert_eq!(model_name, Some("Whisper Tiny".to_string())); + assert_eq!(model_name, Some("Whisper Tiny".to_string()).map(Into::into)); assert_eq!(license, "apache-2.0"); assert_eq!(pipeline_tag, "automatic-speech-recognition"); - assert_eq!(library_name, Some("whisper-apr".to_string())); + assert_eq!(library_name, Some("whisper-apr".to_string()).map(Into::into)); assert_eq!( tags, Some(vec![ "whisper".to_string(), "tiny".to_string(), "asr".to_string() - ]) + ]).map(|v| v.into_iter().map(Into::into).collect()) ); - assert_eq!(message, Some("Initial release".to_string())); + assert_eq!(message, Some("Initial release".to_string()).map(Into::into)); assert!(dry_run); } _ => panic!("Expected Publish command"), diff --git a/crates/apr-cli/src/lib_parse_rosetta_02.rs b/crates/apr-cli/src/lib_parse_rosetta_02.rs index 14ebcd25d8..1ddeea1f83 100644 --- a/crates/apr-cli/src/lib_parse_rosetta_02.rs +++ b/crates/apr-cli/src/lib_parse_rosetta_02.rs @@ -27,9 +27,9 @@ json, } => { assert_eq!(model, PathBuf::from("model.gguf")); - assert_eq!(model_b, Some(PathBuf::from("model2.apr"))); - assert_eq!(output, Some(PathBuf::from("fingerprints.json"))); - assert_eq!(filter, Some("encoder".to_string())); + assert_eq!(model_b, Some(PathBuf::from("model2.apr")).map(Into::into)); + assert_eq!(output, Some(PathBuf::from("fingerprints.json")).map(Into::into)); + assert_eq!(filter, Some("encoder".to_string()).map(Into::into)); assert!(verbose); assert!(json); } @@ -68,8 +68,8 @@ json, } => { assert_eq!(model, PathBuf::from("model.apr")); - assert_eq!(reference, Some(PathBuf::from("ref.gguf"))); - assert_eq!(fingerprints, Some(PathBuf::from("fp.json"))); + assert_eq!(reference, Some(PathBuf::from("ref.gguf")).map(Into::into)); + assert_eq!(fingerprints, Some(PathBuf::from("fp.json")).map(Into::into)); assert!((threshold - 5.0).abs() < f32::EPSILON); assert!(strict); assert!(json); @@ -141,9 +141,9 @@ #[test] fn test_extract_paths_export() { let cmd = Commands::Export { - file: Some(PathBuf::from("model.apr")), - format: "gguf".to_string(), - output: Some(PathBuf::from("out.gguf")), + file: Some(PathBuf::from("model.apr").into()), + format: "gguf".to_string().into(), + output: Some(PathBuf::from("out.gguf").into()), quantize: None, list_formats: false, batch: None, @@ -159,10 +159,10 @@ #[test] fn test_extract_paths_convert() { let cmd = Commands::Convert { - file: PathBuf::from("model.apr"), - quantize: Some("q4k".to_string()), + file: PathBuf::from("model.apr").into(), + quantize: Some("q4k".to_string().into()), compress: None, - output: PathBuf::from("out.apr"), + output: PathBuf::from("out.apr").into(), force: false, }; let paths = extract_model_paths(&cmd); @@ -173,7 +173,7 @@ #[test] fn test_extract_paths_check() { let cmd = Commands::Check { - file: PathBuf::from("model.gguf"), + file: PathBuf::from("model.gguf").into(), no_gpu: false, json: false, }; @@ -185,7 +185,7 @@ #[test] fn test_extract_paths_trace() { let cmd = Commands::Trace { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), layer: None, reference: None, json: false, @@ -195,7 +195,7 @@ interactive: false, save_tensor: None, save_tensor_dir: None, - save_tensor_layers: "0..1".to_string(), + save_tensor_layers: "0..1".to_string().into(), }; let paths = extract_model_paths(&cmd); assert_eq!(paths, vec![PathBuf::from("model.apr")]); @@ -207,9 +207,9 @@ use TestSubcommand; let cmd = Commands::Extended(ExtendedCommands::Test { command: TestSubcommand::Tensor { - file: PathBuf::from("model.apr"), - output: PathBuf::from("./probar-export"), - format: "both".to_string(), + file: PathBuf::from("model.apr").into(), + output: PathBuf::from("./probar-export").into(), + format: "both".to_string().into(), golden: None, layer: None, assert: false, @@ -224,8 +224,8 @@ #[test] fn test_extract_paths_compare_hf() { let cmd = Commands::Extended(ExtendedCommands::CompareHf { - file: PathBuf::from("model.apr"), - hf: "openai/whisper-tiny".to_string(), + file: PathBuf::from("model.apr").into(), + hf: "openai/whisper-tiny".to_string().into(), tensor: None, threshold: 1e-5, json: false, @@ -238,7 +238,7 @@ #[test] fn test_extract_paths_chat() { let cmd = Commands::Extended(ExtendedCommands::Chat { - file: PathBuf::from("model.gguf"), + file: PathBuf::from("model.gguf").into(), temperature: 0.7, top_p: 0.9, max_tokens: 512, @@ -262,8 +262,8 @@ #[test] fn test_extract_paths_eval() { let cmd = Commands::Extended(ExtendedCommands::Eval { - file: PathBuf::from("model.gguf"), - dataset: "wikitext-2".to_string(), + file: PathBuf::from("model.gguf").into(), + dataset: "wikitext-2".to_string().into(), text: None, max_tokens: 512, threshold: 20.0, @@ -284,9 +284,9 @@ #[test] fn test_extract_paths_profile() { let cmd = Commands::Extended(ExtendedCommands::Profile { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), granular: false, - format: "human".to_string(), + format: "human".to_string().into(), focus: None, detect_naive: false, threshold: 10.0, @@ -315,9 +315,9 @@ #[test] fn test_extract_paths_import_hf_url() { let cmd = Commands::Import { - source: "hf://openai/whisper-tiny".to_string(), - output: Some(PathBuf::from("whisper.apr")), - arch: "auto".to_string(), + source: "hf://openai/whisper-tiny".to_string().into(), + output: Some(PathBuf::from("whisper.apr").into()), + arch: "auto".to_string().into(), quantize: None, strict: false, preserve_q4k: false, @@ -336,9 +336,9 @@ #[test] fn test_extract_paths_import_nonexistent_local() { let cmd = Commands::Import { - source: "/tmp/nonexistent_model_abc123.gguf".to_string(), + source: "/tmp/nonexistent_model_abc123.gguf".to_string().into(), output: None, - arch: "auto".to_string(), + arch: "auto".to_string().into(), quantize: None, strict: false, preserve_q4k: false, @@ -357,7 +357,7 @@ #[test] fn test_extract_paths_tui_with_file() { let cmd = Commands::Tui { - file: Some(PathBuf::from("model.apr")), + file: Some(PathBuf::from("model.apr").into()), }; let paths = extract_model_paths(&cmd); assert_eq!(paths, vec![PathBuf::from("model.apr")]); @@ -377,7 +377,7 @@ let cmd = Commands::Extended(ExtendedCommands::Cbtop { model: None, attach: None, - model_path: Some(PathBuf::from("model.gguf")), + model_path: Some(PathBuf::from("model.gguf").into()), headless: false, json: false, output: None, @@ -400,7 +400,7 @@ #[test] fn test_extract_paths_cbtop_no_model_path() { let cmd = Commands::Extended(ExtendedCommands::Cbtop { - model: Some("qwen2.5-coder".to_string()), + model: Some("qwen2.5-coder".to_string().into()), attach: None, model_path: None, headless: false, @@ -425,8 +425,8 @@ #[test] fn test_extract_paths_diff_exempt() { let cmd = Commands::Diff { - file1: PathBuf::from("a.apr"), - file2: PathBuf::from("b.apr"), + file1: PathBuf::from("a.apr").into(), + file2: PathBuf::from("b.apr").into(), weights: false, values: false, filter: None, @@ -445,7 +445,7 @@ #[test] fn test_extract_paths_hex_exempt() { let cmd = Commands::Extended(ExtendedCommands::Hex { - file: PathBuf::from("model.apr"), + file: PathBuf::from("model.apr").into(), tensor: None, limit: 64, stats: false, @@ -457,7 +457,7 @@ contract: false, entropy: false, raw: false, - offset: "0".to_string(), + offset: "0".to_string().into(), width: 16, slice: None, }); diff --git a/crates/apr-cli/src/lib_parse_tensors.rs b/crates/apr-cli/src/lib_parse_tensors.rs index f3d27c4734..09e1a84401 100644 --- a/crates/apr-cli/src/lib_parse_tensors.rs +++ b/crates/apr-cli/src/lib_parse_tensors.rs @@ -24,7 +24,7 @@ } => { assert_eq!(file, PathBuf::from("model.apr")); assert!(stats); - assert_eq!(filter, Some("encoder".to_string())); + assert_eq!(filter, Some("encoder".to_string()).map(Into::into)); assert_eq!(limit, 20); assert!(json); } @@ -44,7 +44,7 @@ tensor, .. } => { - assert_eq!(code_or_file, Some("E001".to_string())); + assert_eq!(code_or_file, Some("E001".to_string()).map(Into::into)); assert!(file.is_none()); assert!(tensor.is_none()); } @@ -72,8 +72,8 @@ .. } => { assert!(code_or_file.is_none()); - assert_eq!(file, Some(PathBuf::from("model.apr"))); - assert_eq!(tensor, Some("embed.weight".to_string())); + assert_eq!(file, Some(PathBuf::from("model.apr")).map(Into::into)); + assert_eq!(tensor, Some("embed.weight".to_string()).map(Into::into)); } _ => panic!("Expected Explain command"), } @@ -112,8 +112,8 @@ save_tensor_layers: _, } => { assert_eq!(file, PathBuf::from("model.apr")); - assert_eq!(layer, Some("layer.0".to_string())); - assert_eq!(reference, Some(PathBuf::from("ref.apr"))); + assert_eq!(layer, Some("layer.0".to_string()).map(Into::into)); + assert_eq!(reference, Some(PathBuf::from("ref.apr")).map(Into::into)); assert!(json); assert!(verbose); assert!(payload); @@ -171,7 +171,7 @@ assert_eq!(file1, PathBuf::from("a.apr")); assert_eq!(file2, PathBuf::from("b.apr")); assert!(values); - assert_eq!(filter, Some("embed".to_string())); + assert_eq!(filter, Some("embed".to_string()).map(Into::into)); assert_eq!(limit, 5); assert!(transpose_aware); assert!(json); @@ -200,7 +200,7 @@ .. } => { assert!(chat); - assert_eq!(prompt, Some("Hello world".to_string())); + assert_eq!(prompt, Some("Hello world".to_string()).map(Into::into)); assert_eq!(source, "model.gguf"); } _ => panic!("Expected Run command"), @@ -248,7 +248,7 @@ .. } => { assert_eq!(source, "model.gguf"); - assert_eq!(positional_prompt, Some("What is 2+2?".to_string())); + assert_eq!(positional_prompt, Some("What is 2+2?".to_string()).map(Into::into)); assert_eq!(prompt, None); } _ => panic!("Expected Run command"), @@ -273,8 +273,8 @@ prompt, .. } => { - assert_eq!(positional_prompt, Some("positional text".to_string())); - assert_eq!(prompt, Some("flag text".to_string())); + assert_eq!(positional_prompt, Some("positional text".to_string()).map(Into::into)); + assert_eq!(prompt, Some("flag text".to_string()).map(Into::into)); } _ => panic!("Expected Run command"), } @@ -293,7 +293,7 @@ .. } => { assert_eq!(source, "model.gguf"); - assert_eq!(positional_prompt, Some("What is 2+2?".to_string())); + assert_eq!(positional_prompt, Some("What is 2+2?".to_string()).map(Into::into)); assert_eq!(max_tokens, 64); } _ => panic!("Expected Run command"), @@ -366,10 +366,10 @@ "Tokenize".to_string(), "Embed".to_string(), "Attention".to_string() - ]) + ]).map(|v| v.into_iter().map(Into::into).collect()) ); assert!(trace_verbose); - assert_eq!(trace_output, Some(PathBuf::from("/tmp/trace.json"))); + assert_eq!(trace_output, Some(PathBuf::from("/tmp/trace.json")).map(Into::into)); assert_eq!(trace_level, "layer"); } _ => panic!("Expected Run command"), diff --git a/crates/apr-cli/src/lib_verbose_inheritance_parse.rs b/crates/apr-cli/src/lib_verbose_inheritance_parse.rs index 16b7bdbf39..2ac1333566 100644 --- a/crates/apr-cli/src/lib_verbose_inheritance_parse.rs +++ b/crates/apr-cli/src/lib_verbose_inheritance_parse.rs @@ -161,7 +161,7 @@ fn test_execute_with_contract_gate_nonexistent() { let cli = Cli { command: Box::new(Commands::Inspect { - file: PathBuf::from("/tmp/nonexistent_contract_test.apr"), + file: PathBuf::from("/tmp/nonexistent_contract_test.apr").into(), vocab: false, filters: false, weights: false, @@ -207,7 +207,7 @@ fn test_execute_rosetta_inspect_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Inspect { - file: PathBuf::from("/tmp/nonexistent_rosetta_inspect.gguf"), + file: PathBuf::from("/tmp/nonexistent_rosetta_inspect.gguf").into(), hexdump: false, json: false, }, @@ -224,8 +224,8 @@ fn test_execute_rosetta_convert_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Convert { - source: PathBuf::from("/tmp/nonexistent_rosetta_convert.gguf"), - target: PathBuf::from("/tmp/out.safetensors"), + source: PathBuf::from("/tmp/nonexistent_rosetta_convert.gguf").into(), + target: PathBuf::from("/tmp/out.safetensors").into(), quantize: None, verify: false, json: false, @@ -244,7 +244,7 @@ fn test_execute_rosetta_fingerprint_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tools(ToolCommands::Rosetta { action: RosettaCommands::Fingerprint { - model: PathBuf::from("/tmp/nonexistent_rosetta_fingerprint.gguf"), + model: PathBuf::from("/tmp/nonexistent_rosetta_fingerprint.gguf").into(), model_b: None, output: None, filter: None, @@ -263,7 +263,7 @@ #[test] fn test_execute_bench_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Bench { - file: PathBuf::from("/tmp/nonexistent_model_bench_test.gguf"), + file: PathBuf::from("/tmp/nonexistent_model_bench_test.gguf").into(), warmup: 1, iterations: 1, max_tokens: 1, @@ -280,8 +280,8 @@ #[test] fn test_execute_eval_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Eval { - file: PathBuf::from("/tmp/nonexistent_model_eval_test.gguf"), - dataset: "wikitext-2".to_string(), + file: PathBuf::from("/tmp/nonexistent_model_eval_test.gguf").into(), + dataset: "wikitext-2".to_string().into(), text: None, max_tokens: 32, threshold: 20.0, @@ -302,9 +302,9 @@ #[test] fn test_execute_profile_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Profile { - file: PathBuf::from("/tmp/nonexistent_model_profile_test.apr"), + file: PathBuf::from("/tmp/nonexistent_model_profile_test.apr").into(), granular: false, - format: "human".to_string(), + format: "human".to_string().into(), focus: None, detect_naive: false, threshold: 10.0, @@ -336,8 +336,8 @@ #[test] fn test_execute_compare_hf_file_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::CompareHf { - file: PathBuf::from("/tmp/nonexistent_model_compare_hf_test.apr"), - hf: "openai/whisper-tiny".to_string(), + file: PathBuf::from("/tmp/nonexistent_model_compare_hf_test.apr").into(), + hf: "openai/whisper-tiny".to_string().into(), tensor: None, threshold: 1e-5, json: false, @@ -354,8 +354,8 @@ fn test_execute_canary_check_file_not_found() { let cli = make_cli(Commands::Canary { command: CanaryCommands::Check { - file: PathBuf::from("/tmp/nonexistent_canary_check.apr"), - canary: PathBuf::from("/tmp/nonexistent_canary.json"), + file: PathBuf::from("/tmp/nonexistent_canary_check.apr").into(), + canary: PathBuf::from("/tmp/nonexistent_canary.json").into(), }, }); let result = execute_command(&cli); @@ -369,11 +369,11 @@ #[test] fn test_execute_publish_dir_not_found() { let cli = make_cli(Commands::Extended(ExtendedCommands::Tools(ToolCommands::Publish { - directory: PathBuf::from("/tmp/nonexistent_publish_dir_test"), - repo_id: "test/test".to_string(), + directory: PathBuf::from("/tmp/nonexistent_publish_dir_test").into(), + repo_id: "test/test".to_string().into(), model_name: None, - license: "mit".to_string(), - pipeline_tag: "text-generation".to_string(), + license: "mit".to_string().into(), + pipeline_tag: "text-generation".to_string().into(), library_name: None, tags: None, message: None, diff --git a/crates/apr-cli/src/main.rs b/crates/apr-cli/src/main.rs index 99536e36a2..02891ac9d0 100644 --- a/crates/apr-cli/src/main.rs +++ b/crates/apr-cli/src/main.rs @@ -1,85 +1,11 @@ //! apr - APR Model Operations CLI //! -//! Entry point shim. See lib.rs for implementation. - -use apr_cli::{execute_command, Cli}; -use clap::Parser; -use colored::control; -use std::process::ExitCode; - -#[cfg(feature = "dhat-heap")] -#[global_allocator] -static ALLOC: dhat::Alloc = dhat::Alloc; - -/// GH-667: Reset SIGPIPE to SIG_DFL so piping to head/less exits cleanly. -#[cfg(unix)] -#[allow(unsafe_code)] -fn reset_sigpipe() { - // SAFETY: signal(SIGPIPE, SIG_DFL) is async-signal-safe per POSIX. - // Called once at program start before any threads are spawned. - unsafe { - libc::signal(libc::SIGPIPE, libc::SIG_DFL); - } -} - -#[cfg(not(unix))] -fn reset_sigpipe() {} - -/// GH-646: Clear FPCR.FZ16 on aarch64 so f16 subnormal scales aren't flushed to zero. -/// The ARM `fcvt s, h` instruction (used by the `half` crate) respects FPCR.FZ16. -/// When set (e.g., by NVIDIA's CUDA runtime on Jetson), subnormal f16 scale values -/// in Q6_K super-blocks are flushed to 0.0, causing `d * scale * q = 0` for all elements. -/// This produces all-zero tensors that pass on x86_64 but fail on aarch64. -#[cfg(target_arch = "aarch64")] -#[allow(unsafe_code)] -fn clear_fz16() { - // SAFETY: Reading/writing FPCR is safe — it only affects floating-point behavior - // for the current thread. Called once at program start before any FP operations. - unsafe { - let fpcr: u64; - core::arch::asm!("mrs {}, fpcr", out(reg) fpcr); - if fpcr & (1 << 19) != 0 { - // FZ16 is set — clear it - let new_fpcr = fpcr & !(1 << 19); - core::arch::asm!("msr fpcr, {}", in(reg) new_fpcr); - } - } -} - -#[cfg(not(target_arch = "aarch64"))] -fn clear_fz16() {} - -fn main() -> ExitCode { - // GH-667: Reset SIGPIPE to default so piping to head/less doesn't panic. - reset_sigpipe(); - // GH-646: Clear FPCR.FZ16 on ARM so f16 subnormals dequantize correctly. - clear_fz16(); - - #[cfg(feature = "dhat-heap")] - let _profiler = dhat::Profiler::new_heap(); - // GH-662: Respect NO_COLOR env var and non-TTY output. - // The `colored` crate's auto-detect doesn't reliably work in all environments. - let no_color = std::env::var("NO_COLOR").is_ok(); - let is_tty = std::io::IsTerminal::is_terminal(&std::io::stdout()); - if no_color || !is_tty { - control::set_override(false); - } - // FALSIFY-GPUTRAIN-007 / INV-GPUTRAIN-007: intercept `apr --version - // --json` BEFORE clap exits on the plain --version flag. Emits the - // 3-key CUDA metadata schema required by gputrain_007. Handler lives - // in lib.rs::emit_version_json to keep the schema next to its tests. - let raw: Vec = std::env::args().collect(); - if raw.iter().any(|a| a == "--version") && raw.iter().any(|a| a == "--json") { - apr_cli::emit_version_json(); - return ExitCode::SUCCESS; - } - - let cli = Cli::parse(); - match execute_command(&cli) { - Ok(()) => ExitCode::SUCCESS, - Err(e) => { - eprintln!("error: {e}"); - e.exit_code() - } - } +//! Entry point shim. The program is `apr_cli::cli_main`, the SAME function the +//! root facade's `src/bin/apr.rs` calls: both packages define a binary named +//! `apr`, and G0.5 (#2582) found the two had been kept equal by hand-copying +//! the prologue, and had already diverged (the dhat allocator existed only +//! here). `entry_point_is_shared` in lib.rs fails if a prologue comes back. + +fn main() -> std::process::ExitCode { + apr_cli::cli_main() } diff --git a/crates/apr-cli/src/model_ops_commands.rs b/crates/apr-cli/src/model_ops_commands.rs index 1319b986b6..f3c50f688e 100644 --- a/crates/apr-cli/src/model_ops_commands.rs +++ b/crates/apr-cli/src/model_ops_commands.rs @@ -6,10 +6,10 @@ pub enum ModelOpsCommands { Finetune { /// Input model file #[arg(value_name = "FILE")] - file: Option, + file: Option, /// Fine-tuning method: auto, full, lora, qlora #[arg(long, short = 'm', default_value = "auto")] - method: String, + method: FreeText, /// LoRA rank (default: auto-selected) #[arg(long, short = 'r')] rank: Option, @@ -21,13 +21,13 @@ pub enum ModelOpsCommands { plan: bool, /// Training data file (JSONL format) #[arg(long, short = 'd', value_name = "FILE")] - data: Option, + data: Option, /// Output path (adapter dir or merged model) #[arg(short, long)] - output: Option, + output: Option, /// Adapter path for merge mode #[arg(long)] - adapter: Option, + adapter: Option, /// Merge adapter into base model #[arg(long)] merge: bool, @@ -42,16 +42,16 @@ pub enum ModelOpsCommands { learning_rate: Option, /// Model size for planning (e.g., "7B", "1.5B") #[arg(long, value_name = "SIZE")] - model_size: Option, + model_size: Option, /// Fine-tuning task: classify (sequence classification) #[arg(long)] - task: Option, + task: Option, /// Number of classes for classification task #[arg(long, default_value = "5")] num_classes: usize, /// Output format for checkpoints: apr, safetensors, or both (comma-separated) #[arg(long, value_name = "FORMAT", default_value = "apr,safetensors")] - checkpoint_format: String, + checkpoint_format: FreeText, /// Oversample minority classes to match majority (for imbalanced datasets) #[arg(long)] oversample: bool, @@ -63,7 +63,7 @@ pub enum ModelOpsCommands { quantize_nf4: bool, /// GPU indices for data-parallel training (e.g., "0,1" for dual GPU) #[arg(long, value_name = "INDICES")] - gpus: Option, + gpus: Option, /// GPU backend selection: auto, cuda, wgpu // #2583: no `value_parser` here meant `--gpu-backend cudaa` parsed and // silently took the `_ =>` ("auto") arm of `gpu_backend_notice`. @@ -72,13 +72,13 @@ pub enum ModelOpsCommands { gpu_backend: String, /// Distributed training role: coordinator or worker #[arg(long, value_name = "ROLE")] - role: Option, + role: Option, /// Address to bind (coordinator) or connect to (worker) #[arg(long, value_name = "ADDR")] - bind: Option, + bind: Option, /// Coordinator address for worker nodes (e.g., "intel:9000") #[arg(long, value_name = "ADDR")] - coordinator: Option, + coordinator: Option, /// Expected number of workers (coordinator only) #[arg(long, value_name = "N")] expect_workers: Option, @@ -89,11 +89,11 @@ pub enum ModelOpsCommands { /// Format: --adapters data/corpus-a.jsonl:checkpoints/adapter-a /// Can be specified multiple times for concurrent adapter training. #[arg(long, value_name = "DATA:CHECKPOINT")] - adapters: Vec, + adapters: Vec, /// Multi-adapter config file: TOML with [[adapter]] entries (GPU-SHARE §2.4) #[arg(long, value_name = "FILE")] - adapters_config: Option, + adapters_config: Option, /// Enable experimental CUDA MPS for concurrent GPU sharing (GPU-SHARE §1.5). /// WARNING: A GPU fault in any MPS client will crash ALL clients on that GPU. @@ -113,10 +113,10 @@ pub enum ModelOpsCommands { Prune { /// Input model file #[arg(value_name = "FILE")] - file: PathBuf, + file: ModelPath, /// Pruning method: magnitude, structured, depth, width, wanda, sparsegpt #[arg(long, short = 'm', default_value = "magnitude")] - method: String, + method: FreeText, /// Target pruning ratio (0-1) #[arg(long, default_value = "0.5")] target_ratio: f32, @@ -125,10 +125,10 @@ pub enum ModelOpsCommands { sparsity: f32, /// Output file path #[arg(short, long)] - output: Option, + output: Option, /// Layers to remove for depth pruning (e.g., "20-24") #[arg(long)] - remove_layers: Option, + remove_layers: Option, /// Analyze mode (identify pruning opportunities) #[arg(long)] analyze: bool, @@ -137,25 +137,25 @@ pub enum ModelOpsCommands { plan: bool, /// Calibration data file #[arg(long, value_name = "FILE")] - calibration: Option, + calibration: Option, }, /// Knowledge distillation (teacher -> student) (GH-247, ALB-011) Distill { /// Teacher model file (positional, for file-based mode) #[arg(value_name = "TEACHER")] - teacher: Option, + teacher: Option, /// Student model file #[arg(long, value_name = "FILE")] - student: Option, + student: Option, /// Training data file #[arg(long, short = 'd', value_name = "FILE")] - data: Option, + data: Option, /// Output file path #[arg(short, long)] - output: Option, + output: Option, /// Distillation strategy: standard, progressive, ensemble #[arg(long, default_value = "standard")] - strategy: String, + strategy: FreeText, /// Temperature for softmax scaling #[arg(long, default_value = "3.0")] temperature: f64, @@ -170,10 +170,10 @@ pub enum ModelOpsCommands { plan: bool, /// YAML config file for two-stage distillation (ALB-011) #[arg(long, value_name = "FILE")] - config: Option, + config: Option, /// Distillation stage: precompute, train (logit KD), or generate (text-based, GH-455) #[arg(long, value_name = "STAGE")] - stage: Option, + stage: Option, /// SPEC-DISTILL-001 Phase 3-prep (PMAT-697): teacher/student backend /// selector. `fixture` (default) uses the in-memory FixtureTeacher /// + FixtureStudent (CPU-only, useful for plumbing tests + CI). @@ -196,6 +196,6 @@ pub enum ModelOpsCommands { /// Phase 4 50K-step dispatch sets this to a tokenized Python /// corpus directory. #[arg(long, value_name = "DIR")] - dataset: Option, + dataset: Option, }, } diff --git a/crates/apr-cli/src/parsing.rs b/crates/apr-cli/src/parsing.rs index cd3e23c326..5e2def1613 100644 --- a/crates/apr-cli/src/parsing.rs +++ b/crates/apr-cli/src/parsing.rs @@ -42,7 +42,7 @@ assert!(print, "-p must still set print"); assert_eq!( model, - Some(PathBuf::from("/tmp/named.gguf")), + Some(PathBuf::from("/tmp/named.gguf")).map(Into::into), "--model written after the prompt must be honoured, not swallowed" ); assert_eq!( @@ -149,7 +149,7 @@ .. } => { assert_eq!(source, "hf://openai/whisper-tiny"); - assert_eq!(prompt, Some("Hello".to_string())); + assert_eq!(prompt, Some("Hello".to_string()).map(Into::into)); assert_eq!(max_tokens, 64); } _ => panic!("Expected Run command"), @@ -481,7 +481,7 @@ } => { assert_eq!(files.len(), 2); assert_eq!(strategy, "weighted"); - assert_eq!(output, Some(PathBuf::from("merged.apr"))); + assert_eq!(output, Some(PathBuf::from("merged.apr")).map(Into::into)); assert_eq!(weights, Some(vec![0.7, 0.3])); } _ => panic!("Expected Merge command"), diff --git a/crates/apr-cli/src/pipeline_commands.rs b/crates/apr-cli/src/pipeline_commands.rs index ee04a38ae8..7a83c5de68 100644 --- a/crates/apr-cli/src/pipeline_commands.rs +++ b/crates/apr-cli/src/pipeline_commands.rs @@ -9,15 +9,15 @@ pub enum PipelineCommands { Plan { /// Path to forjar pipeline manifest (YAML) #[arg(value_name = "MANIFEST")] - manifest: PathBuf, + manifest: ConfigPath, /// Target specific machine #[arg(short, long)] - machine: Option, + machine: Option, /// Filter to resources with this tag #[arg(short, long)] - tag: Option, + tag: Option, /// Show estimated change cost per resource type #[arg(long)] @@ -28,15 +28,15 @@ pub enum PipelineCommands { Apply { /// Path to forjar pipeline manifest (YAML) #[arg(value_name = "MANIFEST")] - manifest: PathBuf, + manifest: ConfigPath, /// Target specific machine #[arg(short, long)] - machine: Option, + machine: Option, /// Filter to resources with this tag #[arg(short, long)] - tag: Option, + tag: Option, /// Number of parallel SSH sessions (default: 5) #[arg(short, long)] @@ -51,13 +51,13 @@ pub enum PipelineCommands { Status { /// Path to forjar pipeline manifest (YAML) #[arg(value_name = "MANIFEST")] - manifest: PathBuf, + manifest: ConfigPath, }, /// Validate pipeline manifest without connecting to machines. Validate { /// Path to forjar pipeline manifest (YAML) #[arg(value_name = "MANIFEST")] - manifest: PathBuf, + manifest: ConfigPath, }, } diff --git a/crates/apr-cli/src/serve_commands.rs b/crates/apr-cli/src/serve_commands.rs index e6f283c435..2332f54f71 100644 --- a/crates/apr-cli/src/serve_commands.rs +++ b/crates/apr-cli/src/serve_commands.rs @@ -17,7 +17,7 @@ pub enum ServeCommands { Plan { /// Model source: local path or HuggingFace repo (hf://org/repo, org/repo) #[arg(value_name = "MODEL")] - model: String, + model: ModelRef, /// Detect GPU via nvidia-smi for VRAM budget #[arg(long)] gpu: bool, @@ -29,25 +29,28 @@ pub enum ServeCommands { seq_len: usize, /// Output format: text, json, yaml #[arg(long, default_value = "text")] - format: String, + format: FreeText, /// Quantization override for HF models (e.g., Q4_K_M, Q6_K, F16) #[arg(long)] - quant: Option, + quant: Option, }, /// Start inference server (REST API, streaming, metrics) + // #3745 S1 v1.1: prompts arrive over HTTP, not through a PromptText arg, + // so the generator is declared by marker (`apr surface` → `generates: true`). + #[command(group(batuta_common::cli_roles::ServesGeneration::group()))] Run { /// Path to model file /// /// Not required with `--list-devices`, which asks what this BUILD can /// dispatch to and needs no model to answer. #[arg(value_name = "FILE", required_unless_present = "list_devices")] - file: Option, + file: Option, /// Port to listen on #[arg(short, long, default_value = "8080")] port: u16, /// Host to bind to #[arg(long, default_value = "127.0.0.1")] - host: String, + host: FreeText, /// Disable CORS #[arg(long)] no_cors: bool, @@ -76,7 +79,7 @@ pub enum ServeCommands { /// Mirrors llama.cpp's `-ngl`, which takes an integer, `auto` or `all` /// and reports what it resolved. Neither comparator has a boolean. #[arg(long, value_name = "N|auto|all|0")] - gpu_layers: Option, + gpu_layers: Option, /// List the accelerators this BUILD can dispatch to, then exit. /// /// Answers "what does this binary actually support" without starting a @@ -91,7 +94,7 @@ pub enum ServeCommands { trace: bool, /// Trace detail level (none, basic, layer) #[arg(long, value_name = "LEVEL", default_value = "basic")] - trace_level: String, + trace_level: FreeText, /// Enable inline Roofline profiling (adds X-Profile headers) #[arg(long)] profile: bool, @@ -107,7 +110,7 @@ pub enum ServeCommands { /// Each request = parent span, each layer = child span with TensorStats. /// Example: --otlp-endpoint http://localhost:4317 #[arg(long, value_name = "URL")] - otlp_endpoint: Option, + otlp_endpoint: Option, /// GH-286: Max context/sequence length for KV cache. Lower = less RSS. #[arg(long, default_value = "4096")] context_length: usize, diff --git a/crates/apr-cli/src/surface.rs b/crates/apr-cli/src/surface.rs new file mode 100644 index 0000000000..931cc9c468 --- /dev/null +++ b/crates/apr-cli/src/surface.rs @@ -0,0 +1,614 @@ +//! `apr surface`: the interface the binary actually has, emitted by the binary +//! (#3745 S1, PMAT-3749). +//! +//! Operator, 2026-09-21, verbatim: "the way to prevent testing leakage is tests +//! are dervived from surface and SHACL enforced...WE ARE NOT ENFORCING our +//! interface AT ALL..it is hand coded not derived". +//! +//! Every release cell used to come from a list someone typed: the ladder's +//! rungs, `{run, chat, serve, code}`, `release_evidence.rs`'s `VERBS`, the +//! hand-maintained `apr-cli-commands-v1.yaml`. A verb, flag or input shape +//! nobody typed was invisible to the gate, and 0.69.0 leaked three that way +//! (`serve` on Qwen3.5 #3571, `code` #3719, `run --prompt P --chat` #3743). +//! +//! This module walks `::command()` and returns every +//! subcommand path and every argument, with the argument's ROLE read from how it +//! is BUILT: +//! +//! * `model` / `prompt` / `input-file` / `other`: the value parser's output type +//! is one of the role types in `batuta_common::cli_roles` (compared by +//! `TypeId`, never by name); +//! * `backend`: the argument belongs to the group the shared [`BackendArg`] +//! flatten creates (`::group_id()`); +//! * `mode`: structural, meaning a flag action or a finite value set; +//! * `unknown`: a free-form value (raw `PathBuf`, or raw `String` without a +//! finite value set) that no role type declares. [`unknown_outside_foreign`] +//! is the marker guard: it must be empty. +//! +//! The JSON (`apr-cli-surface/v1`, agreed on #3745) is produced from the binary +//! under test at gate time and is never committed. [`emit`] is the library entry +//! point that in-process gates call; `apr surface --json` is a thin wrapper +//! over it, and a test pins the two equal. + +use std::any::TypeId; +use std::path::PathBuf; + +use batuta_common::cli_roles::{self, Carrier, Role}; +use clap::{ArgAction, CommandFactory}; +use serde::Serialize; + +use crate::{BackendArg, Cli}; + +/// The schema string consumers refuse to read if it is not one they know. +/// +/// v1.1 (cop ruling on #3745, after S2 found `serve run`'s prompts arrive over +/// HTTP) adds `commands[].generates`. +pub const SCHEMA: &str = "apr-cli-surface/v1.1"; + +/// Stack for the recursive walk. Building and walking the full `apr` clap tree +/// overflows a default test-thread stack (see the note at +/// `commands/tokenize.rs` on FALSIFY-APR-TOK-PAR-004), so [`emit`] always runs +/// it on a thread of its own. +const WALK_STACK: usize = 64 * 1024 * 1024; + +/// The whole surface: one entry per subcommand path. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct Surface { + /// Always [`SCHEMA`]. + pub schema: &'static str, + /// The binary this surface was read from. + pub binary: BinaryId, + /// Every role an argument can carry, in the order the schema lists them. + pub roles: Vec<&'static str>, + /// Arguments the root declares with `global = true`; clap propagates each + /// of them to every command path, and they are not repeated per command. + pub global_args: Vec, + /// Every subcommand path, nested and group nodes included, sorted by path. + pub commands: Vec, +} + +/// Name, version and source revision of the binary under test. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct BinaryId { + pub name: String, + pub version: String, + pub git_sha: String, +} + +/// One subcommand path. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct CommandEntry { + /// The subcommand names from the root, e.g. `["serve", "run"]`. + pub path: Vec, + /// `path` joined with single spaces: the key #3739's CRUX mapping uses. + pub key: String, + /// Other names clap accepts for this subcommand (visible and hidden). + pub aliases: Vec, + /// Absent from `--help`. + pub hidden: bool, + /// Declared by another crate's `Subcommand` type rather than by apr-cli. + pub foreign: bool, + /// True iff the command has no subcommands. + pub leaf: bool, + /// The command generates text from prompts (v1.1): it has a `PromptText` + /// argument, or it carries the `ServesGeneration` marker because its + /// prompts arrive another way (`serve run` over HTTP, `chat` on stdin, + /// `mcp` as JSON-RPC). Read from how the command is built, never its name. + pub generates: bool, + /// Names of the direct subcommands, in declaration order. + pub subcommands: Vec, + /// Arguments declared on this command (not the propagated globals). + pub args: Vec, +} + +/// One argument. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct ArgEntry { + pub id: String, + /// Long name, without dashes. + pub long: Option, + pub short: Option, + /// Every other long name clap accepts (visible and hidden). + pub long_aliases: Vec, + /// Every other short name clap accepts (visible and hidden). + pub short_aliases: Vec, + pub positional: bool, + /// 1-based position of a positional argument. + pub index: Option, + pub required: bool, + pub takes_value: bool, + pub num_args: NumArgs, + /// `flag`, `count`, `enum`, `path`, `text`, `int`, `float` or `other`. + pub value_type: &'static str, + /// The finite set of accepted values (hidden values excluded), else empty. + pub values: Vec, + pub default: Vec, + /// One of [`Surface::roles`]. + pub role: &'static str, + /// The role type that built this argument (`ModelPath`, `BackendArg`, …). + pub marker: Option<&'static str>, + /// For role `sampling` (v1.1): `seed`, `temperature`, `top_k`, `top_p`, + /// `min_p`, `repeat_penalty` or `repeat_last_n`, read from the sampling + /// group the argument joined, never from its name. `null` otherwise. + pub sampling_kind: Option<&'static str>, + pub hidden: bool, + /// Ids of the arguments this one conflicts with. + pub conflicts_with: Vec, +} + +/// How many values one occurrence takes. `max: None` is unbounded. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] +pub struct NumArgs { + pub min: usize, + pub max: Option, +} + +impl Surface { + /// The versioned JSON document `apr surface --json` prints. + #[must_use] + pub fn to_json(&self) -> String { + serde_json::to_string_pretty(self).expect("surface is plain data and always serializes") + } + + /// The entry for a subcommand path, e.g. `["serve", "run"]`. + #[must_use] + pub fn command(&self, path: &[&str]) -> Option<&CommandEntry> { + self.commands + .iter() + .find(|c| c.path.iter().map(String::as_str).eq(path.iter().copied())) + } +} + +impl CommandEntry { + /// The argument with this id. + #[must_use] + pub fn arg(&self, id: &str) -> Option<&ArgEntry> { + self.args.iter().find(|a| a.id == id) + } +} + +/// The surface of the `apr` binary this library is linked into. +/// +/// Runs the walk on its own thread with an explicit stack (see [`WALK_STACK`]), +/// so it is safe to call from a test thread or from `main`. +#[must_use] +pub fn emit() -> Surface { + std::thread::Builder::new() + .name("apr-surface".to_string()) + .stack_size(WALK_STACK) + .spawn(|| emit_from(&Cli::command())) + .expect("spawn the surface walk thread") + .join() + .expect("the surface walk panicked") +} + +/// The surface of an arbitrary clap tree rooted at `root`. +/// +/// Membership (which commands and arguments exist) is read from `root` as the +/// code declared it. Finalised attributes (`num_args`, positional indices, +/// conflicts) are read from a BUILT clone. Building also adds clap's generated +/// `help` subcommand and the `--help`/`--version` flags; those are clap's +/// surface, not apr's, so they come from nowhere in the unbuilt tree and are +/// not emitted. +/// +/// Call this on a thread with a large stack when `root` is the full `apr` tree. +#[must_use] +pub fn emit_from(root: &clap::Command) -> Surface { + let mut built = root.clone(); + built.build(); + let foreign = foreign_mounts(root).mounts; + + let global_args = root + .get_arguments() + .filter(|a| a.is_global_set()) + .map(|a| arg_entry(a, &built, root)) + .collect(); + + let mut commands = Vec::new(); + for sub in root.get_subcommands() { + let built_sub = child(&built, sub.get_name()); + walk(sub, built_sub, &mut vec![], &foreign, false, &mut commands); + } + commands.sort_by(|a, b| a.path.cmp(&b.path)); + + Surface { + schema: SCHEMA, + binary: BinaryId { + name: root.get_name().to_string(), + version: env!("CARGO_PKG_VERSION").to_string(), + git_sha: env!("APR_GIT_SHA").to_string(), + }, + roles: ROLES.iter().map(|r| r.as_str()).collect(), + global_args, + commands, + } +} + +/// Every role, in schema order. +const ROLES: [Role; 8] = [ + Role::Model, + Role::Prompt, + Role::InputFile, + Role::Backend, + Role::Sampling, + Role::Mode, + Role::Other, + Role::Unknown, +]; + +fn child<'a>(built: &'a clap::Command, name: &str) -> &'a clap::Command { + built + .get_subcommands() + .find(|s| s.get_name() == name) + .expect("a built tree has every subcommand its unbuilt source declares") +} + +fn walk( + cmd: &clap::Command, + built: &clap::Command, + parent: &mut Vec, + foreign: &[Vec], + inherited_foreign: bool, + out: &mut Vec, +) { + let mut path = parent.clone(); + path.push(cmd.get_name().to_string()); + let is_foreign = inherited_foreign || foreign.iter().any(|m| *m == path); + + let subcommands: Vec = cmd + .get_subcommands() + .map(|s| s.get_name().to_string()) + .collect(); + let args: Vec = cmd + .get_arguments() + .filter(|a| !a.is_global_set()) + .map(|a| arg_entry(a, built, cmd)) + .collect(); + out.push(CommandEntry { + key: path.join(" "), + path: path.clone(), + aliases: cmd.get_all_aliases().map(str::to_string).collect(), + hidden: cmd.is_hide_set(), + foreign: is_foreign, + leaf: subcommands.is_empty(), + generates: generates(cmd, &args), + subcommands, + args, + }); + + for sub in cmd.get_subcommands() { + walk( + sub, + child(built, sub.get_name()), + &mut path, + foreign, + is_foreign, + out, + ); + } +} + +fn arg_entry(arg: &clap::Arg, built_cmd: &clap::Command, cmd: &clap::Command) -> ArgEntry { + let built_arg = built_cmd + .get_arguments() + .find(|b| b.get_id() == arg.get_id()) + .expect("a built command keeps every argument its source declares"); + let class = classify(arg, cmd, built_cmd); + let takes_value = built_arg.get_action().takes_values(); + let num_args = built_arg + .get_num_args() + .map(|r| NumArgs { + min: r.min_values(), + max: (r.max_values() != usize::MAX).then_some(r.max_values()), + }) + .unwrap_or(if takes_value { + NumArgs { + min: 1, + max: Some(1), + } + } else { + NumArgs { + min: 0, + max: Some(0), + } + }); + ArgEntry { + id: arg.get_id().to_string(), + long: arg.get_long().map(str::to_string), + short: arg.get_short(), + long_aliases: arg + .get_all_aliases() + .unwrap_or_default() + .into_iter() + .map(str::to_string) + .collect(), + short_aliases: arg.get_all_short_aliases().unwrap_or_default(), + positional: arg.is_positional(), + index: built_arg.get_index(), + required: arg.is_required_set(), + takes_value, + num_args, + value_type: class.value_type, + values: finite_values(arg), + default: arg + .get_default_values() + .iter() + .map(|v| v.to_string_lossy().into_owned()) + .collect(), + role: class.role.as_str(), + marker: class.marker, + sampling_kind: cli_roles::SamplingArg::kind_of(built_cmd, built_arg) + .map(cli_roles::SamplingKind::as_str), + hidden: arg.is_hide_set(), + conflicts_with: built_cmd + .get_arg_conflicts_with(built_arg) + .into_iter() + .map(|c| c.get_id().to_string()) + .collect(), + } +} + +/// True iff the command generates from prompts: a `PromptText` argument, or the +/// command-level `ServesGeneration` marker. Both are read from construction. +fn generates(cmd: &clap::Command, args: &[ArgEntry]) -> bool { + cli_roles::ServesGeneration::is_on(cmd) || args.iter().any(|a| a.role == Role::Prompt.as_str()) +} + +/// An argument's role, the type that declared it, and the shape of its value. +struct Class { + role: Role, + marker: Option<&'static str>, + value_type: &'static str, +} + +/// Decide an argument's role from how it is built. Nothing here reads a name. +fn classify(arg: &clap::Arg, cmd: &clap::Command, built: &clap::Command) -> Class { + let ty = arg.get_value_parser().type_id(); + let finite = !finite_values(arg).is_empty(); + let value_type = match arg.get_action() { + ArgAction::SetTrue | ArgAction::SetFalse => "flag", + ArgAction::Count => "count", + _ if finite => "enum", + _ => scalar_type(arg.get_value_parser()), + }; + + if in_backend_group(arg, cmd) { + return Class { + role: Role::Backend, + marker: Some("BackendArg"), + value_type, + }; + } + // v1.1: membership in the SamplingArg group, which is visible only on the + // BUILT command (clap merges `#[arg(group = …)]` there). + if cli_roles::SamplingArg::kind_of(built, arg).is_some() { + return Class { + role: Role::Sampling, + marker: Some("SamplingArg"), + value_type, + }; + } + if let Some(m) = cli_roles::MARKERS.iter().find(|m| ty == m.type_id()) { + let value_type = match m.carrier { + Carrier::Path => "path", + Carrier::Text => "text", + }; + return Class { + role: m.role, + marker: Some(m.name), + value_type, + }; + } + let role = match value_type { + "flag" | "count" | "enum" => Role::Mode, + "path" | "text" => Role::Unknown, + _ => Role::Other, + }; + Class { + role, + marker: None, + value_type, + } +} + +/// `path`/`text` for the two raw free-form types, `int`/`float` for numbers, +/// `other` for anything else a value parser can yield. +fn scalar_type(parser: &clap::builder::ValueParser) -> &'static str { + let ty = parser.type_id(); + let is = |t: TypeId| ty == t; + if is(TypeId::of::()) { + "path" + } else if is(TypeId::of::()) { + "text" + } else if [ + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + TypeId::of::(), + ] + .into_iter() + .any(is) + { + "int" + } else if is(TypeId::of::()) || is(TypeId::of::()) { + "float" + } else { + "other" + } +} + +/// Values the argument's parser declares as its whole domain. clap's bool +/// parser lists `true`/`false`, which is a finite domain and so a mode. +fn finite_values(arg: &clap::Arg) -> Vec { + arg.get_possible_values() + .iter() + .filter(|v| !v.is_hide_set()) + .map(|v| v.get_name().to_string()) + .collect() +} + +/// True iff `arg` is declared through the shared [`BackendArg`] flatten. The +/// group id comes from the type, through `clap::Args`, never from a literal. +fn in_backend_group(arg: &clap::Arg, cmd: &clap::Command) -> bool { + let Some(group) = ::group_id() else { + return false; + }; + cmd.get_groups() + .any(|g| g.get_id() == &group && g.get_args().any(|a| a == arg.get_id())) +} + +/// The subtrees of `root` that another crate's `Subcommand` type declares. +/// +/// apr embeds these CLIs whole (`apr pv …` is `aprender-contracts-cli`'s own +/// enum). Their free-form arguments cannot all carry apr's role types yet, so +/// they are emitted as `foreign: true` and a remaining untyped argument there +/// is role `unknown`, which S2 charges a runs-or-refuses cell and ratchets +/// shrink-only (cop ruling on #3745). +/// +/// A mount is found by TYPE: each listed type renders its own subcommand set, +/// and a subtree of `root` whose children are exactly that set (order ignored) +/// is its mount. +/// The list fails closed both ways. A foreign CLI missing from it has its +/// untyped arguments counted as apr-cli's, which the marker guard refuses. A +/// listed type that is mounted zero times or more than once is reported in +/// [`ForeignScan::problems`], which the marker guard requires to be empty for +/// the real `apr` tree, so a stale entry cannot sit in the list unnoticed. +fn foreign_mounts(root: &clap::Command) -> ForeignScan { + let probes: [(&str, clap::Command); 7] = [ + ( + "aprender_contracts_cli::cli::Commands", + probe::(), + ), + ( + "alimentar::cli::Commands", + probe::(), + ), + ("simular::cli::Commands", probe::()), + ( + "aprender_rag_cli::Commands", + probe::(), + ), + ( + "aprender_zram_cli::Commands", + probe::(), + ), + ("cgp::cli::Commands", probe::()), + // `apr ptx-debug` mounts the former `aprender-ptx-debug` binary's enum (#4062). + ( + "trueno_ptx_debug::cli::Command", + probe::(), + ), + ]; + let mut scan = ForeignScan::default(); + for (ty, p) in &probes { + // A set, not a sequence: declaration order is not identity, and clap's + // own `mut_subcommand` moves the child it mutates to the end. + let mut want: Vec<&str> = p.get_subcommands().map(clap::Command::get_name).collect(); + want.sort_unstable(); + let mut hits = Vec::new(); + find_mounts(root, &mut Vec::new(), &want, &mut hits); + if hits.len() != 1 { + scan.problems.push(format!( + "foreign CLI `{ty}` must be mounted exactly once in the tree, found {} mount(s): {hits:?}", + hits.len() + )); + } + scan.mounts.extend(hits); + } + scan +} + +/// Where the foreign CLIs are mounted, and what is wrong with the list. +#[derive(Debug, Default)] +pub(crate) struct ForeignScan { + /// Path of each mount point. + pub(crate) mounts: Vec>, + /// A listed type mounted zero times, or more than once. + pub(crate) problems: Vec, +} + +/// The foreign-list problems for the real `apr` tree. The marker guard requires +/// this to be empty. +#[must_use] +pub fn foreign_list_problems() -> Vec { + std::thread::Builder::new() + .stack_size(WALK_STACK) + .spawn(|| foreign_mounts(&Cli::command()).problems) + .expect("spawn") + .join() + .expect("join") +} + +fn probe() -> clap::Command { + T::augment_subcommands(clap::Command::new("probe")) +} + +fn find_mounts( + cmd: &clap::Command, + path: &mut Vec, + want: &[&str], + hits: &mut Vec>, +) { + for sub in cmd.get_subcommands() { + path.push(sub.get_name().to_string()); + let mut names: Vec<&str> = sub.get_subcommands().map(clap::Command::get_name).collect(); + names.sort_unstable(); + if !names.is_empty() && names == want { + hits.push(path.clone()); + } + find_mounts(sub, path, want, hits); + path.pop(); + } +} + +/// Every argument outside a foreign subtree whose role is `unknown`, as +/// `" "`. This is the marker guard's reading, and it must be empty: +/// a free-form argument that no role type declares is how a model or prompt +/// argument hides from the cells derived from this surface. +#[must_use] +pub fn unknown_outside_foreign(surface: &Surface) -> Vec { + let mut out: Vec = surface + .global_args + .iter() + .filter(|a| a.role == Role::Unknown.as_str()) + .map(|a| format!(" {}", a.id)) + .collect(); + for c in surface.commands.iter().filter(|c| !c.foreign) { + for a in c.args.iter().filter(|a| a.role == Role::Unknown.as_str()) { + out.push(format!("{} {}", c.key, a.id)); + } + } + out +} + +/// `apr surface`: print [`emit`]'s JSON and nothing else on stdout. +/// +/// Written with `std::io::Write`, not the crate's shadowed `println!`, so +/// `--quiet` cannot suppress the only thing the verb exists to produce. +pub(crate) fn run() -> Result<(), crate::CliError> { + write_surface(&mut std::io::stdout().lock()) + .map_err(|e| crate::CliError::ValidationFailed(format!("apr surface: writing stdout: {e}"))) +} + +/// The bytes `apr surface` writes: [`emit`]'s JSON and one newline. +fn write_surface(out: &mut impl std::io::Write) -> std::io::Result<()> { + out.write_all(emit().to_json().as_bytes())?; + out.write_all(b"\n")?; + out.flush() +} + +#[cfg(test)] +#[path = "surface_tests.rs"] +mod tests; + +#[cfg(test)] +#[path = "surface_guard_tests.rs"] +mod guard; diff --git a/crates/apr-cli/src/surface_guard_tests.rs b/crates/apr-cli/src/surface_guard_tests.rs new file mode 100644 index 0000000000..9e07cb6185 --- /dev/null +++ b/crates/apr-cli/src/surface_guard_tests.rs @@ -0,0 +1,323 @@ +//! #3745 S1.3: the marker guard. +//! +//! Issue #3745 mutant 3, verbatim: "a command that loads a model with a plain +//! `PathBuf` arg instead of `ModelArg` → the marker guard is RED." +//! +//! The guard does not try to detect model loading; apr-cli has no loader funnel +//! to detect it at (about 13 loader APIs across realizar and aprender-core), and +//! a list of loaders would rot like the verb lists #3745 removes. Instead it +//! makes an undeclared role inexpressible: outside a foreign subtree, every +//! free-form argument (raw `PathBuf`, or raw `String` without a finite value +//! set) is RED until it is built through a role type. A model argument can +//! therefore only be written down as a model argument, or as a typed lie that +//! is visible in the diff. + +use super::*; +use clap::Arg; + +fn big_stack(f: impl FnOnce() -> T + Send + 'static) -> T { + std::thread::Builder::new() + .stack_size(WALK_STACK) + .spawn(f) + .expect("spawn") + .join() + .expect("join") +} + +const REMEDY: &str = "declare each through a role type from batuta_common::cli_roles \ + (ModelPath/ModelRef for a model, PromptText for a prompt, InputFile for input data, \ + OutputPath/DirPath/ConfigPath/FreeText otherwise) — a raw PathBuf/String hides the \ + argument's role from the cells derived from `apr surface` (#3745)"; + +/// The guard itself, on the tree this binary ships. +#[test] +fn no_free_form_arg_is_undeclared_outside_foreign_subtrees() { + let unknown = big_stack(|| unknown_outside_foreign(&emit())); + assert!( + unknown.is_empty(), + "{} free-form argument(s) with no declared role:\n {}\n{REMEDY}", + unknown.len(), + unknown.join("\n ") + ); +} + +/// The foreign-CLI list fails closed: every listed type is mounted exactly once. +#[test] +fn the_foreign_list_matches_the_tree() { + let problems = foreign_list_problems(); + assert!(problems.is_empty(), "{}", problems.join("\n")); +} + +/// The verbs whose leaks #3745 names, and the two foreign model arguments the +/// cop ruled into S1, carry the roles their cells are derived from. +#[test] +fn the_leak_verbs_declare_model_prompt_and_input_roles() { + let s = big_stack(emit); + let role = |path: &[&str], id: &str| { + s.command(path) + .and_then(|c| c.arg(id)) + .map(|a| a.role) + .unwrap_or_else(|| panic!("{path:?} {id} is on the surface")) + }; + // #3743: `run --prompt P --chat` vs `-i file --chat`. + assert_eq!(role(&["run"], "source"), "model"); + assert_eq!(role(&["run"], "prompt"), "prompt"); + assert_eq!(role(&["run"], "positional_prompt"), "prompt"); + assert_eq!(role(&["run"], "input"), "input-file"); + assert_eq!(role(&["chat"], "file"), "model"); + // #3571: `apr serve` on Qwen3.5. + assert_eq!(role(&["serve", "run"], "file"), "model"); + // #3719: `apr code`'s backend is `apr serve`. + assert_eq!(role(&["code"], "model"), "model"); + assert_eq!(role(&["code"], "prompt"), "prompt"); + // Cop ruling on #3745: foreign, but typed in S1 itself. + assert_eq!(role(&["pv", "verify-structure"], "model"), "model"); + assert_eq!(role(&["rag", "transcribe"], "model"), "model"); +} + +// ── mutants, planted in the real tree on every run ────────────────────────── + +fn unknown_after( + mutate: impl FnOnce(clap::Command) -> clap::Command + Send + 'static, +) -> Vec { + big_stack(move || unknown_outside_foreign(&emit_from(&mutate(Cli::command())))) +} + +/// Mutant 3a: `inspect`'s model argument rebuilt as a plain `PathBuf`. +#[test] +fn mutant_an_existing_model_arg_reverted_to_plain_pathbuf_is_red() { + let unknown = unknown_after(|root| { + root.mut_subcommand("inspect", |c| { + c.mut_arg("file", |a| a.value_parser(clap::value_parser!(PathBuf))) + }) + }); + assert_eq!( + unknown, + ["inspect file"], + "the guard must name exactly the reverted argument" + ); +} + +/// Mutant 3b: a new command that takes its model through a plain `PathBuf`. +#[test] +fn mutant_a_new_command_with_a_plain_pathbuf_model_is_red() { + let unknown = unknown_after(|root| { + root.subcommand( + clap::Command::new("zz-new-verb").arg( + Arg::new("model") + .long("model") + .value_parser(clap::value_parser!(PathBuf)), + ), + ) + }); + assert_eq!(unknown, ["zz-new-verb model"]); +} + +/// Mutant 3c: a prompt taken as a plain `String` (the #3743 class). +#[test] +fn mutant_a_prompt_as_plain_string_is_red() { + let unknown = unknown_after(|root| { + root.mut_subcommand("run", |c| { + c.mut_arg("prompt", |a| a.value_parser(clap::value_parser!(String))) + }) + }); + assert_eq!(unknown, ["run prompt"]); +} + +/// A raw argument in a nested apr-cli subcommand is caught as well. +#[test] +fn mutant_a_nested_plain_pathbuf_is_red() { + let unknown = unknown_after(|root| { + root.mut_subcommand("serve", |serve| { + serve.mut_subcommand("run", |c| { + c.mut_arg("file", |a| a.value_parser(clap::value_parser!(PathBuf))) + }) + }) + }); + assert_eq!(unknown, ["serve run file"]); +} + +/// A raw global argument is caught: globals reach every command path. +#[test] +fn mutant_a_plain_global_is_red() { + let unknown = unknown_after(|root| { + root.arg( + Arg::new("zz-config") + .long("zz-config") + .global(true) + .value_parser(clap::value_parser!(PathBuf)), + ) + }); + assert_eq!(unknown, [" zz-config"]); +} + +/// Inside a foreign subtree a raw argument is emitted as `unknown` (S2 owes +/// its verb a runs-or-refuses cell and ratchets the count), and it is not the +/// marker guard's to refuse. (Reverting THIS argument is still caught, by +/// `the_leak_verbs_declare_model_prompt_and_input_roles`.) +#[test] +fn a_foreign_raw_arg_is_unknown_not_red() { + let s = big_stack(|| { + emit_from(&Cli::command().mut_subcommand("pv", |pv| { + pv.mut_subcommand("verify-structure", |c| { + c.mut_arg("model", |a| a.value_parser(clap::value_parser!(PathBuf))) + }) + })) + }); + let arg = s + .command(&["pv", "verify-structure"]) + .and_then(|c| c.arg("model")) + .expect("pv verify-structure --model"); + assert_eq!(arg.role, "unknown"); + assert!( + unknown_outside_foreign(&s).is_empty(), + "foreign subtrees are not the marker guard's" + ); +} + +/// v1.1: the generators S2 derives prompt-shape cells for. `serve run`, `chat` +/// and `mcp` take their prompts over HTTP, stdin and JSON-RPC, so only the +/// `ServesGeneration` marker makes them generators; `run` and `code` are by +/// their `PromptText` args. Model tools that never generate are not. +#[test] +fn the_generators_are_declared() { + let s = big_stack(emit); + let generates = |path: &[&str]| { + s.command(path) + .map(|c| c.generates) + .unwrap_or_else(|| panic!("{path:?} is on the surface")) + }; + for path in [ + &["serve", "run"][..], + &["chat"], + &["mcp"], + &["run"], + &["code"], + ] { + assert!(generates(path), "{path:?} must be a generator"); + } + for path in [ + &["inspect"][..], + &["tensors"], + &["serve", "plan"], + &["surface"], + // Encoders and scorers take EncodeText, not PromptText (aprender-97): + // thinking × context-rung cells mean nothing for them. + &["embed"], + &["rerank"], + &["eval"], + // Tokenizes its --prompt and never runs the weights (#3749 rebase). + &["tokenize", "encode"], + ] { + assert!(!generates(path), "{path:?} does not generate"); + } +} + +/// Mutant: `serve run` with its ServesGeneration marker dropped. Its real +/// argument set, rebuilt without the marker, reads `generates: false`. So the +/// marker, not any argument, is what makes it a generator (#3571's verb). +#[test] +fn mutant_serve_run_without_its_marker_does_not_generate() { + let (real, stripped) = big_stack(|| { + let root = Cli::command(); + let run = root + .find_subcommand("serve") + .and_then(|s| s.find_subcommand("run")) + .expect("serve run") + .clone(); + let mut bare = clap::Command::new("run"); + for a in run.get_arguments() { + bare = bare.arg(a.clone()); + } + let real = emit_from(&clap::Command::new("apr").subcommand(run)); + let stripped = emit_from(&clap::Command::new("apr").subcommand(bare)); + (real, stripped) + }); + assert!(real.command(&["run"]).expect("run").generates); + assert!( + !stripped.command(&["run"]).expect("run").generates, + "without the marker serve run must read generates=false — the marker is load-bearing" + ); +} + +/// v1.1 `sampling` (for S2.5's flag-effect sampling controls): the generation +/// sampling controls carry the SamplingArg marker. Numbers with the same names +/// that are not generation sampling (distillation temperature, data seeds, +/// rerank's top-k) are not. +#[test] +fn the_sampling_controls_are_declared() { + let s = big_stack(emit); + let role = |path: &[&str], id: &str| { + s.command(path) + .and_then(|c| c.arg(id)) + .map(|a| a.role) + .unwrap_or_else(|| panic!("{path:?} {id} is on the surface")) + }; + let kind = |path: &[&str], id: &str| { + s.command(path) + .and_then(|c| c.arg(id)) + .and_then(|a| a.sampling_kind) + }; + // `run`'s field ids happen to equal their kinds' spellings; the kind is read + // from the group each field joined, and the case table below the name level + // (`ungrouped_temperature`) shows a name alone yields no kind. + for (id, want) in [ + ("temperature", "temperature"), + ("top_k", "top_k"), + ("top_p", "top_p"), + ("seed", "seed"), + ("repeat_penalty", "repeat_penalty"), + ("repeat_last_n", "repeat_last_n"), + ] { + assert_eq!(role(&["run"], id), "sampling", "run --{id}"); + assert_eq!(kind(&["run"], id), Some(want), "run --{id} sampling_kind"); + } + assert_eq!(kind(&["chat"], "top_p"), Some("top_p")); + assert_eq!( + kind(&["distill"], "temperature"), + None, + "KD temperature has no kind" + ); + assert_eq!(role(&["chat"], "temperature"), "sampling"); + assert_eq!(role(&["chat"], "top_p"), "sampling"); + assert_eq!( + role(&["rosetta", "compare-inference"], "temperature"), + "sampling" + ); + assert_ne!( + role(&["distill"], "temperature"), + "sampling", + "KD temperature" + ); + assert_ne!(role(&["data", "split"], "seed"), "sampling", "a data seed"); + assert_ne!(role(&["rerank"], "top_k"), "sampling", "a ranking cut-off"); +} + +/// Mutant: `run --temperature` with its SamplingArg marker dropped reads as an +/// ordinary number, so `the_sampling_controls_are_declared` would go RED. +#[test] +fn mutant_temperature_without_its_marker_is_not_sampling() { + let s = big_stack(|| { + emit_from(&Cli::command().mut_subcommand("run", |c| { + c.mut_arg("temperature", |a| { + a.group(clap::builder::Resettable::::Reset) + }) + })) + }); + let t = s + .command(&["run"]) + .and_then(|c| c.arg("temperature")) + .expect("run --temperature"); + assert_ne!( + t.role, "sampling", + "without the marker the role must not be sampling" + ); + let k = s + .command(&["run"]) + .and_then(|c| c.arg("top_k")) + .expect("top_k"); + assert_eq!( + k.role, "sampling", + "only the mutated argument loses its role" + ); +} diff --git a/crates/apr-cli/src/surface_tests.rs b/crates/apr-cli/src/surface_tests.rs new file mode 100644 index 0000000000..c4d7e9f5dc --- /dev/null +++ b/crates/apr-cli/src/surface_tests.rs @@ -0,0 +1,390 @@ +//! #3745 S1: the surface emitter, read back. + +use super::*; +use clap::{Arg, ArgAction}; + +/// Run `f` on a thread with the walk's stack. The full `apr` tree does not fit +/// a default test-thread stack. +fn big_stack(f: impl FnOnce() -> T + Send + 'static) -> T { + std::thread::Builder::new() + .stack_size(WALK_STACK) + .spawn(f) + .expect("spawn") + .join() + .expect("join") +} + +/// S1.1: a flag added to the clap tree appears in the JSON, under its command +/// path, with a role. The emitter reads the tree it is given; it keeps no list +/// that a new flag could be missing from. +#[test] +fn the_emitter_sees_a_flag_added_to_the_clap_tree() { + let (before, after) = big_stack(|| { + let root = Cli::command(); + let before = emit_from(&root); + let mutated = root.mut_subcommand("run", |run| { + run.arg( + Arg::new("zz-dummy") + .long("zz-dummy") + .action(ArgAction::SetTrue), + ) + }); + (before, emit_from(&mutated)) + }); + let run_before = before.command(&["run"]).expect("run is on the surface"); + assert!( + run_before.arg("zz-dummy").is_none(), + "the dummy flag must be absent before it is added" + ); + + let run = after.command(&["run"]).expect("run is on the surface"); + let dummy = run + .arg("zz-dummy") + .expect("a flag added to the clap tree must appear in the surface"); + assert_eq!(dummy.long.as_deref(), Some("zz-dummy")); + assert_eq!(dummy.value_type, "flag"); + assert_eq!(dummy.role, "mode"); + let json = after.to_json(); + assert!( + json.contains("\"zz-dummy\""), + "the dummy flag must be in the JSON text too" + ); +} + +#[test] +fn the_surface_is_versioned_and_deterministic() { + let (a, b) = big_stack(|| (emit().to_json(), emit().to_json())); + assert_eq!(a, b, "two emissions from one binary must be byte-identical"); + let v: serde_json::Value = serde_json::from_str(&a).expect("the surface is JSON"); + assert_eq!(v["schema"], SCHEMA); + assert_eq!(v["schema"], "apr-cli-surface/v1.1"); + assert_eq!(v["binary"]["name"], "apr"); + let roles: Vec<&str> = v["roles"] + .as_array() + .expect("roles") + .iter() + .filter_map(|r| r.as_str()) + .collect(); + assert_eq!( + roles, + [ + "model", + "prompt", + "input-file", + "backend", + "sampling", + "mode", + "other", + "unknown" + ] + ); + let paths: Vec> = v["commands"] + .as_array() + .expect("commands") + .iter() + .map(|c| serde_json::from_value(c["path"].clone()).expect("path is a string array")) + .collect(); + let mut sorted = paths.clone(); + sorted.sort(); + assert_eq!(paths, sorted, "commands are sorted by path"); +} + +/// clap's generated `help` subcommand and `--help`/`--version` flags are not +/// apr's declared surface; emitting them would owe a release cell per group. +#[test] +fn clap_generated_help_and_version_are_not_emitted() { + let s = big_stack(emit); + // `help` is never an apr-declared id. `version` can be (`data x registry + // delete --version` names a registry entry's version), so the generated + // `--version` is checked where apr declares no such flag: the root and `run`. + for c in &s.commands { + assert!( + c.args.iter().all(|a| a.id != "help"), + "`{} --help` is clap-generated", + c.key + ); + } + assert!(s + .global_args + .iter() + .all(|a| a.id != "help" && a.id != "version")); + assert!( + s.command(&["run"]).expect("run").arg("version").is_none(), + "propagated --version" + ); + // `serve` has subcommands and does not disable clap's help subcommand. + let serve = s.command(&["serve"]).expect("serve"); + assert!(!serve.leaf); + assert!( + s.command(&["serve", "help"]).is_none(), + "clap's generated `serve help` must not be emitted" + ); +} + +#[test] +fn globals_are_emitted_once_at_the_root() { + let s = big_stack(emit); + let ids: Vec<&str> = s.global_args.iter().map(|a| a.id.as_str()).collect(); + assert!( + ids.contains(&"json") && ids.contains(&"quiet") && ids.contains(&"verbose"), + "{ids:?}" + ); + let run = s.command(&["run"]).expect("run"); + assert!( + run.arg("json").is_none(), + "a propagated global is not repeated per command" + ); +} + +#[test] +fn the_backend_role_comes_from_the_backend_arg_group() { + let s = big_stack(emit); + for path in [&["run"][..], &["chat"][..], &["serve", "run"][..]] { + let c = s + .command(path) + .unwrap_or_else(|| panic!("{path:?} is on the surface")); + let b = c + .arg("backend") + .unwrap_or_else(|| panic!("{path:?} has --backend")); + assert_eq!(b.role, "backend", "{path:?} --backend"); + assert_eq!(b.marker, Some("BackendArg")); + assert_eq!( + b.values, + crate::BACKEND_VALUES.to_vec(), + "{path:?} --backend values" + ); + } +} + +/// The `surface` verb itself is hidden from `--help` but present on the surface, +/// so S3 derives it as a verb and S2 owes it a runs cell. +#[test] +fn the_surface_verb_is_hidden_and_listed() { + let s = big_stack(emit); + let c = s.command(&["surface"]).expect("surface lists itself"); + assert!(c.hidden); + assert!(c.leaf); + assert!(!c.foreign); +} + +/// Every foreign CLI the walker lists is found mounted exactly once, and every +/// command under a mount is marked foreign (the mount point included). +#[test] +fn foreign_mounts_are_found_by_type() { + let s = big_stack(emit); + for mount in [ + &["pv"][..], + &["data", "x"][..], + &["sim"][..], + &["rag"][..], + &["zram"][..], + &["cgp"][..], + ] { + let c = s + .command(mount) + .unwrap_or_else(|| panic!("{mount:?} is on the surface")); + assert!(c.foreign, "{mount:?} is a foreign mount"); + for sub in &c.subcommands { + let mut p: Vec<&str> = mount.to_vec(); + p.push(sub); + assert!( + s.command(&p).is_some_and(|e| e.foreign), + "{p:?} is under a foreign mount" + ); + } + } + let data = s.command(&["data"]).expect("data"); + assert!( + !data.foreign, + "`data` is apr-cli's own; only its `x` arm is alimentar's" + ); + assert!(!s.command(&["run"]).expect("run").foreign); +} + +// ── classification case table ──────────────────────────────────────────────── + +/// One row per way an argument can be built. `expect` is (value_type, role, +/// marker). The table is the specification of `classify`: a class of argument +/// with no row here is a class nobody decided. +#[test] +fn classify_case_table() { + use batuta_common::cli_roles::{ + ConfigPath, DirPath, EncodeText, FreeText, InputFile, ModelPath, ModelRef, OutputPath, + PromptText, + }; + #[derive(clap::Parser)] + #[command(groups(batuta_common::cli_roles::SamplingArg::groups()))] + struct Probe { + // v1.1: a sampling control, by group membership; its twin with no group + // is an ordinary number. + #[arg(long, group = batuta_common::cli_roles::SamplingKind::Temperature.id())] + temperature: Option, + #[arg(long, group = batuta_common::cli_roles::SamplingKind::TopK.id())] + top_k: Option, + #[arg(long)] + ungrouped_temperature: Option, + model_path: ModelPath, + #[arg(long)] + model_ref: Option, + #[arg(long)] + prompt: Option, + #[arg(long)] + input: Option, + #[arg(long)] + output: Option, + #[arg(long)] + dir: Option, + #[arg(long)] + config: Option, + #[arg(long)] + name: Option, + #[arg(long)] + encoded: Option, + #[arg(long)] + raw_path: Option, + #[arg(long)] + raw_text: Option, + #[arg(long, value_parser = ["a", "b"])] + listed: Option, + #[arg(long)] + flag: bool, + #[arg(long, action = ArgAction::Count)] + level: u8, + #[arg(long)] + n: Option, + #[arg(long)] + t: Option, + #[arg(long, value_enum)] + fmt: Option, + #[arg(long)] + c: Option, + #[command(flatten)] + backend: BackendArg, + } + let rows: &[(&str, (&str, &str, Option<&str>))] = &[ + ("model_path", ("path", "model", Some("ModelPath"))), + ("model_ref", ("text", "model", Some("ModelRef"))), + ("prompt", ("text", "prompt", Some("PromptText"))), + ("input", ("path", "input-file", Some("InputFile"))), + ("output", ("path", "other", Some("OutputPath"))), + ("dir", ("path", "other", Some("DirPath"))), + ("config", ("path", "other", Some("ConfigPath"))), + ("name", ("text", "other", Some("FreeText"))), + // Encoder/scorer input: a model input, but not a prompt (v1.1). + ("encoded", ("text", "other", Some("EncodeText"))), + // The two rows the marker guard exists for. + ("raw_path", ("path", "unknown", None)), + ("raw_text", ("text", "unknown", None)), + ("listed", ("enum", "mode", None)), + ("flag", ("flag", "mode", None)), + ("level", ("count", "mode", None)), + ("n", ("int", "other", None)), + ("t", ("float", "other", None)), + ("fmt", ("enum", "mode", None)), + ("c", ("other", "other", None)), + ("backend", ("enum", "backend", Some("BackendArg"))), + ("temperature", ("float", "sampling", Some("SamplingArg"))), + ("top_k", ("int", "sampling", Some("SamplingArg"))), + // The name says "temperature"; without the group it is not sampling. + ("ungrouped_temperature", ("float", "other", None)), + ]; + let cmd = ::command(); + let root = clap::Command::new("apr").subcommand(cmd.name("probe")); + let s = emit_from(&root); + let probe = s.command(&["probe"]).expect("probe"); + assert_eq!( + probe.args.len(), + rows.len(), + "every arg of the probe has a row" + ); + for (id, (value_type, role, marker)) in rows { + let a = probe.arg(id).unwrap_or_else(|| panic!("probe has {id}")); + assert_eq!( + (a.value_type, a.role, a.marker), + (*value_type, *role, *marker), + "row {id}" + ); + } +} + +/// The verb writes [`emit`]'s document and nothing else. The binary-level pin +/// (the spawned `apr surface --json` equals `emit()`) is the integration test +/// `tests/surface_binary_pin.rs`. +#[test] +fn the_verb_writes_exactly_what_emit_returns() { + let (written, emitted) = big_stack(|| { + let mut buf = Vec::new(); + write_surface(&mut buf).expect("write to a Vec"); + (buf, emit().to_json()) + }); + assert_eq!(written, format!("{emitted}\n").into_bytes()); +} + +/// `apr surface` parses to the hidden verb, with and without `--json`/`--quiet`. +#[test] +fn apr_surface_parses_to_the_surface_verb() { + use clap::Parser; + for argv in [ + &["apr", "surface"][..], + &["apr", "surface", "--json"][..], + &["apr", "surface", "--quiet"][..], + ] { + let cli = big_stack({ + let argv: Vec = argv.iter().map(|s| s.to_string()).collect(); + move || { + Cli::try_parse_from(argv).map(|c| matches!(*c.command, crate::Commands::Surface)) + } + }); + assert_eq!( + cli.ok(), + Some(true), + "{argv:?} must parse to Commands::Surface" + ); + } +} + +/// v1.1 `generates`: one row per way a command can be built. A command +/// generates iff it has a `PromptText` argument or carries the +/// `ServesGeneration` marker; a name — the arg's or the group's — never counts. +#[test] +fn generates_case_table() { + use batuta_common::cli_roles::{ModelPath, PromptText, ServesGeneration}; + let model = || Arg::new("model").value_parser(clap::value_parser!(ModelPath)); + let prompt_arg = + |p: clap::builder::ValueParser| Arg::new("prompt").long("prompt").value_parser(p); + let root = clap::Command::new("apr") + .subcommand( + clap::Command::new("with-prompt") + .arg(model()) + .arg(prompt_arg(clap::value_parser!(PromptText).into())), + ) + .subcommand( + clap::Command::new("marked") + .arg(model()) + .group(ServesGeneration::group()), + ) + .subcommand(clap::Command::new("model-only").arg(model())) + .subcommand( + clap::Command::new("lookalike-group") + .arg(model()) + .group(clap::ArgGroup::new("ServesGeneration").multiple(true)), + ) + .subcommand( + clap::Command::new("raw-prompt-name") + .arg(model()) + .arg(prompt_arg(clap::value_parser!(String))), + ); + let s = emit_from(&root); + let rows = [ + ("with-prompt", true), + ("marked", true), + ("model-only", false), + ("lookalike-group", false), + ("raw-prompt-name", false), + ]; + assert_eq!(s.commands.len(), rows.len(), "every command has a row"); + for (name, want) in rows { + let c = s.command(&[name]).unwrap_or_else(|| panic!("{name}")); + assert_eq!(c.generates, want, "row {name}"); + } +} diff --git a/crates/apr-cli/src/tokenize_commands.rs b/crates/apr-cli/src/tokenize_commands.rs index 08cf198b51..587749661c 100644 --- a/crates/apr-cli/src/tokenize_commands.rs +++ b/crates/apr-cli/src/tokenize_commands.rs @@ -14,19 +14,19 @@ pub enum TokenizeCommands { Plan { /// Path to training corpus (text file, one document per line) #[arg(long, value_name = "FILE")] - data: PathBuf, + data: InputFile, /// Target vocabulary size #[arg(long, default_value = "32000")] vocab_size: usize, /// Tokenizer algorithm: bpe, wordpiece, unigram #[arg(long, default_value = "bpe")] - algorithm: String, + algorithm: FreeText, /// Output directory for trained tokenizer #[arg(short, long, default_value = "./tokenizer-output")] - output: PathBuf, + output: OutputPath, /// Output format: text, json, yaml #[arg(long, default_value = "text")] - format: String, + format: FreeText, }, /// Train a tokenizer on the corpus. @@ -38,16 +38,16 @@ pub enum TokenizeCommands { Apply { /// Path to training corpus (text file, one document per line) #[arg(long, value_name = "FILE")] - data: PathBuf, + data: InputFile, /// Target vocabulary size #[arg(long, default_value = "32000")] vocab_size: usize, /// Tokenizer algorithm: bpe, wordpiece, unigram #[arg(long, default_value = "bpe")] - algorithm: String, + algorithm: FreeText, /// Output directory for trained tokenizer #[arg(short, long, default_value = "./tokenizer-output")] - output: PathBuf, + output: OutputPath, /// Maximum number of lines to read from corpus (0 = all) #[arg(long, default_value = "0")] max_lines: usize, @@ -64,7 +64,7 @@ pub enum TokenizeCommands { /// Path to corpus: a `.jsonl` file or a directory containing `.jsonl` files. /// Each line must be a JSON object with a `content` field. #[arg(long, value_name = "PATH")] - corpus: PathBuf, + corpus: InputFile, /// Target vocabulary size. Default 50_257 matches GPT-2 convention /// (50_000 BPE merges + 256 byte-level fallback tokens + 1 sentinel) /// and the MODEL-2 albor tokenizer contract (tokenizer-bpe-v1 v1.2.0). @@ -78,10 +78,10 @@ pub enum TokenizeCommands { min_frequency: usize, /// Output directory; will contain vocab.json and merges.txt. #[arg(long, default_value = "./tokenizer-output")] - output: PathBuf, + output: OutputPath, /// Unicode normalization form applied to each document before training. #[arg(long, default_value = "nfc")] - normalization: String, + normalization: FreeText, }, /// Import a HuggingFace tokenizer.json into aprender's two-file @@ -104,10 +104,10 @@ pub enum TokenizeCommands { ImportHf { /// Path to input HuggingFace tokenizer.json (BPE model required). #[arg(long, value_name = "FILE")] - input: PathBuf, + input: InputFile, /// Output directory; will contain vocab.json + merges.txt + manifest.json. #[arg(long, value_name = "DIR")] - output: PathBuf, + output: OutputPath, /// Include `added_tokens` in vocab.json (default: BPE state machine only). /// Use this when the downstream consumer needs special tokens (e.g., /// `<|im_start|>`, `<|endoftext|>`) materialized in vocab.json itself. @@ -125,13 +125,13 @@ pub enum TokenizeCommands { Encode { /// GGUF model file whose tokenizer to run #[arg(value_name = "MODEL")] - model: PathBuf, + model: ModelPath, /// Text to encode #[arg(short = 'p', long, value_name = "TEXT", conflicts_with = "file")] - prompt: Option, + prompt: Option, /// File whose contents to encode (read as UTF-8, verbatim) #[arg(short = 'f', long, value_name = "FILE")] - file: Option, + file: Option, }, /// Encode a JSONL corpus into `.bin` shards per contracts/pretokenize-bin-v1.yaml. /// @@ -154,25 +154,25 @@ pub enum TokenizeCommands { /// are encoded in command-line order and shard numbering is /// continuous across sources. #[arg(long, value_name = "PATH", required = true)] - corpus: Vec, + corpus: Vec, /// Directory containing vocab.json + merges.txt from `apr tokenize train`. #[arg(long, value_name = "DIR")] - tokenizer: PathBuf, + tokenizer: InputFile, /// Output directory for shard-NNNN.bin + manifest.json. #[arg(long, value_name = "DIR")] - output: PathBuf, + output: OutputPath, /// Target tokens per shard (shard closes once this limit is reached). #[arg(long, default_value = "10000000")] shard_tokens: usize, /// JSONL field to encode (default: `content`). #[arg(long, default_value = "content")] - content_field: String, + content_field: FreeText, /// Unicode normalization (must match tokenizer training). #[arg(long, default_value = "nfc")] - normalization: String, + normalization: FreeText, /// EOS insertion policy: none|between|after. #[arg(long, default_value = "between")] - eos_policy: String, + eos_policy: FreeText, /// Number of rayon workers for per-document BPE encoding. /// /// Defaults to `std::thread::available_parallelism()` (logical CPU count). @@ -240,13 +240,13 @@ pub enum TokenizeCommands { /// Output directory containing shard-NNNN.bin files. /// `manifest.json` will be written into this directory. #[arg(long, value_name = "DIR")] - output: PathBuf, + output: OutputPath, /// Optional tokenizer directory; when provided, `vocab.json` /// is read for the manifest's `vocab_size` field. Without it, /// `vocab_size` is recorded as `null` (provenance-incomplete /// but otherwise valid). #[arg(long, value_name = "DIR")] - tokenizer: Option, + tokenizer: Option, /// Emit the manifest body as JSON to stdout (in addition to /// writing to disk). #[arg(long, default_value_t = false)] diff --git a/crates/apr-cli/src/tool_commands.rs b/crates/apr-cli/src/tool_commands.rs index e18ff68307..11ce2264b9 100644 --- a/crates/apr-cli/src/tool_commands.rs +++ b/crates/apr-cli/src/tool_commands.rs @@ -8,16 +8,16 @@ pub enum ToolCommands { auto_verify: bool, /// Run specific step #[arg(long)] - step: Option, + step: Option, /// Model tier: tiny (0.5B), small (1.5B), medium (7B), large (32B) #[arg(long, default_value = "small")] - tier: String, + tier: FreeText, /// Model directory #[arg(long, default_value = "./models")] - model_dir: PathBuf, + model_dir: DirPath, /// Baselines to compare: llama-cpp,ollama #[arg(long, default_value = "llama-cpp,ollama")] - baseline: String, + baseline: FreeText, /// Enable ZRAM compression #[arg(long)] zram: bool, @@ -46,28 +46,28 @@ pub enum ToolCommands { Publish { /// Directory containing model files to publish #[arg(value_name = "DIRECTORY")] - directory: PathBuf, + directory: DirPath, /// HuggingFace repository ID (e.g., paiml/whisper-apr-tiny) #[arg(value_name = "REPO_ID")] - repo_id: String, + repo_id: FreeText, /// Model display name #[arg(long)] - model_name: Option, + model_name: Option, /// License (SPDX identifier, default: mit) #[arg(long, default_value = "mit")] - license: String, + license: FreeText, /// Pipeline tag (e.g., automatic-speech-recognition, text-generation) #[arg(long, default_value = "text-generation")] - pipeline_tag: String, + pipeline_tag: FreeText, /// Library name (e.g., whisper-apr, aprender) #[arg(long)] - library_name: Option, + library_name: Option, /// Additional tags (comma-separated) #[arg(long, value_delimiter = ',')] - tags: Option>, + tags: Option>, /// Commit message #[arg(long)] - message: Option, + message: Option, /// Dry run (preview without uploading) #[arg(long)] dry_run: bool, @@ -84,13 +84,13 @@ pub enum ToolCommands { Oracle { /// Model file path or hf:// URI #[arg(value_name = "SOURCE")] - source: Option, + source: Option, /// Show contract for a model family (e.g., qwen2, llama, whisper, bert) #[arg(long)] - family: Option, + family: Option, /// Filter to a specific size variant (e.g., 0.5b, 7b) #[arg(long)] - size: Option, + size: Option, /// Run full contract compliance check #[arg(long)] compliance: bool, @@ -121,15 +121,15 @@ pub enum ToolCommands { Encrypt { /// Path to model file (safetensors, GGUF, etc.) #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Output path for encrypted file #[arg(short, long, value_name = "FILE")] - output: PathBuf, + output: OutputPath, /// Encryption key file (32 bytes). If omitted, reads passphrase from stdin. #[arg(long, value_name = "FILE")] - key_file: Option, + key_file: Option, /// Force overwrite if output file exists #[arg(long)] @@ -140,15 +140,15 @@ pub enum ToolCommands { Decrypt { /// Path to encrypted `.enc` file #[arg(value_name = "FILE")] - file: PathBuf, + file: InputFile, /// Output path for decrypted model #[arg(short, long, value_name = "FILE")] - output: PathBuf, + output: OutputPath, /// Decryption key file (32 bytes). If omitted, reads passphrase from stdin. #[arg(long, value_name = "FILE")] - key_file: Option, + key_file: Option, /// Force overwrite if output file exists #[arg(long)] diff --git a/crates/apr-cli/src/train_commands.rs b/crates/apr-cli/src/train_commands.rs index d32063a009..3ffda0bcc6 100644 --- a/crates/apr-cli/src/train_commands.rs +++ b/crates/apr-cli/src/train_commands.rs @@ -14,29 +14,29 @@ pub enum TrainCommands { Plan { /// Path to training data (JSONL). Only read by `apr finetune --task classify`. #[arg(long, value_name = "FILE")] - data: Option, + data: Option, /// Model size: "0.5B", "9B", "7B", "13B" #[arg(long, default_value = "0.5B")] - model_size: String, + model_size: FreeText, /// Path to model weights directory #[arg(long, value_name = "DIR")] - model_path: Option, + model_path: Option, /// Number of output classes #[arg(long, default_value = "5")] num_classes: usize, /// Task type: pretrain (causal LM). Classification fine-tuning is /// `apr finetune --task classify`, not this command. #[arg(long, default_value = "pretrain")] - task: String, + task: FreeText, /// YAML training config (for --task pretrain) #[arg(long, value_name = "FILE")] - config: Option, + config: Option, /// Output directory for checkpoints #[arg(short, long, default_value = "/tmp/training-output")] - output: PathBuf, + output: OutputPath, /// HPO strategy: tpe, grid, random, manual #[arg(long, default_value = "tpe")] - strategy: String, + strategy: FreeText, /// HPO budget (number of trials) #[arg(long, default_value = "20")] budget: usize, @@ -57,13 +57,13 @@ pub enum TrainCommands { batch_size: Option, /// Validation data file (JSONL) #[arg(long, value_name = "FILE")] - val_data: Option, + val_data: Option, /// Test data file (JSONL) #[arg(long, value_name = "FILE")] - test_data: Option, + test_data: Option, /// Output format: text, json, yaml #[arg(long, default_value = "text")] - format: String, + format: FreeText, }, /// Execute a training plan (allocate GPU, run trials). @@ -76,27 +76,27 @@ pub enum TrainCommands { Apply { /// Path to a saved plan file (YAML or JSON from `apr train plan`) #[arg(long, value_name = "FILE")] - plan: Option, + plan: Option, /// YAML training config (for --task pretrain) #[arg(long, value_name = "FILE")] - config: Option, + config: Option, /// Task type: pretrain (causal LM). Classification fine-tuning is /// `apr finetune --task classify`, not this command. #[arg(long, default_value = "pretrain")] - task: String, + task: FreeText, // ── Inline plan params (used when no --plan file is given) ───── /// Path to training data (JSONL) #[arg(long, value_name = "FILE")] - data: Option, + data: Option, /// Model size: "0.5B", "9B", "7B", "13B" #[arg(long, default_value = "0.5B")] - model_size: String, + model_size: FreeText, /// Path to model weights directory #[arg(long, value_name = "DIR")] - model_path: Option, + model_path: Option, /// Number of output classes #[arg(long, default_value = "5")] num_classes: usize, @@ -106,10 +106,10 @@ pub enum TrainCommands { /// When omitted, the config's `training.output_dir` is used (default /// `./checkpoints`). The directory is created if it does not exist. #[arg(short, long, value_name = "DIR")] - output: Option, + output: Option, /// HPO strategy: tpe, grid, random, manual #[arg(long, default_value = "tpe")] - strategy: String, + strategy: FreeText, /// HPO budget (number of trials) #[arg(long, default_value = "20")] budget: usize, @@ -141,7 +141,7 @@ pub enum TrainCommands { rank: Option, /// Coordinator address for distributed training (default: 0.0.0.0:9000) #[arg(long, value_name = "HOST:PORT")] - coordinator_addr: Option, + coordinator_addr: Option, // ── Reproducibility params (R-084 C-DETERM-001) ── /// Enable bitwise deterministic training (CUBLAS_WORKSPACE_CONFIG, cuDNN deterministic) @@ -172,7 +172,7 @@ pub enum TrainCommands { Watch { /// YAML training config to run and watch #[arg(long, value_name = "FILE")] - config: PathBuf, + config: ConfigPath, /// Maximum number of restart attempts #[arg(long, default_value = "5")] @@ -201,11 +201,11 @@ pub enum TrainCommands { Sweep { /// Base YAML training config to sweep from #[arg(long, value_name = "FILE")] - config: PathBuf, + config: ConfigPath, /// Search strategy: grid or random #[arg(long, default_value = "random")] - strategy: String, + strategy: FreeText, /// Number of configs to generate (random) or max combinations (grid) #[arg(long, default_value = "10")] @@ -213,7 +213,7 @@ pub enum TrainCommands { /// Output directory for generated configs #[arg(long, default_value = "sweeps/")] - output_dir: PathBuf, + output_dir: OutputPath, /// Seed for random search reproducibility #[arg(long, default_value = "42")] @@ -232,7 +232,7 @@ pub enum TrainCommands { Halving { /// Directory containing sweep-*.yaml configs (from `apr train sweep`) #[arg(long, value_name = "DIR")] - sweep_dir: PathBuf, + sweep_dir: DirPath, /// Number of halving rounds (default: 3) #[arg(long, default_value = "3")] @@ -252,7 +252,7 @@ pub enum TrainCommands { /// Output JSON file for results #[arg(long, default_value = "sweeps/hpo-results.json")] - output: PathBuf, + output: OutputPath, }, /// Archive a checkpoint into a release bundle. @@ -262,19 +262,19 @@ pub enum TrainCommands { Archive { /// Path to checkpoint directory #[arg(value_name = "CHECKPOINT_DIR")] - checkpoint_dir: PathBuf, + checkpoint_dir: ModelPath, /// Output archive directory #[arg(short, long, value_name = "DIR")] - output: PathBuf, + output: OutputPath, /// Release version tag (e.g., "v1.0") #[arg(long = "release-version")] - release_version: Option, + release_version: Option, /// Release notes #[arg(long)] - notes: Option, + notes: Option, }, /// Submit multi-adapter training jobs to a cluster (GPU-SHARE Phase 3). @@ -284,15 +284,15 @@ pub enum TrainCommands { Submit { /// Path to cluster config YAML #[arg(long, value_name = "FILE")] - cluster: PathBuf, + cluster: ConfigPath, /// Model checkpoint path (.apr) #[arg(long, value_name = "FILE")] - model: PathBuf, + model: ModelPath, /// Adapter specs: DATA:CHECKPOINT pairs (one per adapter) #[arg(long = "adapter", value_name = "DATA:CHECKPOINT")] - adapters: Vec, + adapters: Vec, /// LoRA rank #[arg(long, default_value = "16")] @@ -318,6 +318,6 @@ pub enum TrainCommands { ClusterStatus { /// Path to cluster config YAML #[arg(long, value_name = "FILE")] - cluster: PathBuf, + cluster: ConfigPath, }, } diff --git a/crates/apr-cli/src/validate.rs b/crates/apr-cli/src/validate.rs index 3651c634dd..586a17c28a 100644 --- a/crates/apr-cli/src/validate.rs +++ b/crates/apr-cli/src/validate.rs @@ -11,7 +11,7 @@ fn extract_extended_model_paths(command: &ExtendedCommands) -> Vec { // active subcommand variant. ExtendedCommands::Test { command: probar_sub } => match probar_sub { TestSubcommand::Tensor { file, .. } => { - vec![file.clone()] + vec![file.to_path_buf()] }, // `llm` measures an HTTP endpoint, not a local model file. Listed // explicitly so a future subcommand that DOES name a file has to @@ -22,19 +22,19 @@ fn extract_extended_model_paths(command: &ExtendedCommands) -> Vec { | ExtendedCommands::Chat { file, .. } | ExtendedCommands::Bench { file, .. } | ExtendedCommands::Eval { file, .. } - | ExtendedCommands::Profile { file, .. } => vec![file.clone()], + | ExtendedCommands::Profile { file, .. } => vec![file.to_path_buf()], - ExtendedCommands::Cbtop { model_path, .. } => model_path.iter().cloned().collect(), + ExtendedCommands::Cbtop { model_path, .. } => model_path.iter().map(|p| p.to_path_buf()).collect(), // Rosetta action subcommands ExtendedCommands::Tools(ToolCommands::Rosetta { action }) => match action { RosettaCommands::Convert { source, .. } | RosettaCommands::Chain { source, .. } - | RosettaCommands::Verify { source, .. } => vec![source.clone()], + | RosettaCommands::Verify { source, .. } => vec![source.to_path_buf()], RosettaCommands::CompareInference { model_a, model_b, .. } => { - vec![model_a.clone(), model_b.clone()] + vec![model_a.to_path_buf(), model_b.to_path_buf()] } // Diagnostic rosetta commands — exempt _ => vec![], @@ -64,7 +64,7 @@ fn extract_model_paths(command: &Commands) -> Vec { } } Commands::Export { plan: true, .. } => vec![], - Commands::Export { file, .. } => file.iter().cloned().collect(), + Commands::Export { file, .. } => file.iter().map(|p| p.to_path_buf()).collect(), // GH-471: Serve is exempt from pre-dispatch contract validation. // The model loader itself fails fast on corrupt files. // Contract gate was loading 17 GB+ models just to check tensor counts — @@ -72,22 +72,22 @@ fn extract_model_paths(command: &Commands) -> Vec { Commands::Serve { .. } => vec![], Commands::Trace { file, .. } | Commands::Convert { file, .. } - | Commands::Check { file, .. } => vec![file.clone()], + | Commands::Check { file, .. } => vec![file.to_path_buf()], Commands::Merge { plan: true, .. } => vec![], - Commands::Merge { files, .. } => files.clone(), + Commands::Merge { files, .. } => files.iter().map(|p| p.to_path_buf()).collect(), - Commands::Quantize { file, .. } => vec![file.clone()], - Commands::ModelOps(ModelOpsCommands::Prune { file, .. }) => vec![file.clone()], + Commands::Quantize { file, .. } => vec![file.to_path_buf()], + Commands::ModelOps(ModelOpsCommands::Prune { file, .. }) => vec![file.to_path_buf()], Commands::ModelOps(ModelOpsCommands::Distill { teacher, .. }) => { - teacher.iter().cloned().collect() + teacher.iter().map(|p| p.to_path_buf()).collect() } #[cfg(feature = "training")] Commands::ModelOps(ModelOpsCommands::Finetune { file, .. }) => { - file.iter().cloned().collect() + file.iter().map(|p| p.to_path_buf()).collect() } - Commands::Tui { file, .. } => file.iter().cloned().collect(), + Commands::Tui { file, .. } => file.iter().map(|p| p.to_path_buf()).collect(), Commands::Import { source, .. } => { let path = PathBuf::from(source); if path.exists() { diff --git a/crates/apr-cli/tests/cli_commands.rs b/crates/apr-cli/tests/cli_commands.rs index d5386dd867..ceb7aadac6 100644 --- a/crates/apr-cli/tests/cli_commands.rs +++ b/crates/apr-cli/tests/cli_commands.rs @@ -86,6 +86,7 @@ fn registered_commands() -> Vec<&'static str> { "gpu", "profile", "ptx", + "ptx-debug", "ptx-map", "cbtop", "data", diff --git a/crates/apr-cli/tests/qwen3_moe_apr_run_live_falsifier.rs b/crates/apr-cli/tests/qwen3_moe_apr_run_live_falsifier.rs index 7510e7bf3c..0de253371e 100644 --- a/crates/apr-cli/tests/qwen3_moe_apr_run_live_falsifier.rs +++ b/crates/apr-cli/tests/qwen3_moe_apr_run_live_falsifier.rs @@ -24,6 +24,10 @@ //! development host has the GGUF mmapped at //! `/home/noah/.cache/pacha/models/2b88b180a790988f.gguf`. //! +//! A lane that sets `APR_MOE_FALSIFY_MODEL=` gets no skip path: the +//! test runs against exactly that file and panics `UNMEASURABLE` if it is +//! missing (#4179). +//! //! ## Why this is heavy //! //! `apr run --max-tokens 1` on the 17.3 GB Q4_K_M GGUF performs: @@ -52,10 +56,23 @@ const FRESH_PROMPT: &str = "M32c.2.2.2.1.4 live falsifier 2026-04-29: write the #[test] fn f_qw3_moe_c22214_001_apr_run_emits_at_least_one_non_whitespace_char() { - let Some(gguf_path) = CANONICAL_QWEN3_CODER_GGUF_PATHS - .iter() - .find(|p| Path::new(p).exists()) - else { + // A lane that names the model is a lane that measures it: a missing file + // there is RED, never a vacuous SKIP (#4179, no-defer doctrine). + let required = std::env::var("APR_MOE_FALSIFY_MODEL") + .ok() + .filter(|p| !p.is_empty()); + if let Some(p) = &required { + assert!( + Path::new(p).is_file(), + "F-QW3-MOE-C22214-001: UNMEASURABLE — APR_MOE_FALSIFY_MODEL={p} does not exist" + ); + } + let Some(gguf_path) = required.as_deref().or_else(|| { + CANONICAL_QWEN3_CODER_GGUF_PATHS + .iter() + .copied() + .find(|p| Path::new(p).exists()) + }) else { eprintln!( "F-QW3-MOE-C22214-001: SKIP — no cached Qwen3-Coder GGUF at any of {:?}", CANONICAL_QWEN3_CODER_GGUF_PATHS diff --git a/crates/apr-cli/tests/surface_binary_pin.rs b/crates/apr-cli/tests/surface_binary_pin.rs new file mode 100644 index 0000000000..335ea07ccc --- /dev/null +++ b/crates/apr-cli/tests/surface_binary_pin.rs @@ -0,0 +1,58 @@ +//! #3745 S1: `apr surface --json` from the built binary is exactly the document +//! the library's `surface::emit()` returns. +//! +//! In-process gates (S3's hand-list guard, S2's cell derivation in lib tests) +//! read `emit()`. The release gate reads the release-candidate binary's +//! `apr surface --json`. If the two could differ, a gate would pass on one +//! surface and ship the other. + +use std::process::Command; + +fn apr(args: &[&str]) -> std::process::Output { + Command::new(env!("CARGO_BIN_EXE_apr")) + .args(args) + .output() + .expect("spawn the apr binary") +} + +#[test] +fn the_binary_prints_the_library_surface() { + let expected = format!("{}\n", apr_cli::surface::emit().to_json()); + for argv in [ + &["surface", "--json"][..], + &["surface"][..], + &["surface", "--quiet"][..], + ] { + let out = apr(argv); + assert!( + out.status.success(), + "`apr {}` exited {:?}; stderr:\n{}", + argv.join(" "), + out.status.code(), + String::from_utf8_lossy(&out.stderr) + ); + let stdout = String::from_utf8(out.stdout).expect("utf-8 JSON"); + assert!( + stdout == expected, + "`apr {}` stdout differs from surface::emit() ({} vs {} bytes)", + argv.join(" "), + stdout.len(), + expected.len() + ); + let v: serde_json::Value = + serde_json::from_str(&stdout).expect("stdout is exactly one JSON document"); + assert_eq!(v["schema"], "apr-cli-surface/v1.1"); + } +} + +/// Hidden means absent from `--help`, not absent from the binary. +#[test] +fn surface_is_hidden_from_help() { + let out = apr(&["--help"]); + assert!(out.status.success()); + let help = String::from_utf8_lossy(&out.stdout); + assert!( + !help.lines().any(|l| l.trim_start().starts_with("surface ")), + "`surface` must not be listed in `apr --help`" + ); +} diff --git a/crates/aprender-build-sha/Cargo.toml b/crates/aprender-build-sha/Cargo.toml new file mode 100644 index 0000000000..5647c1d3f7 --- /dev/null +++ b/crates/aprender-build-sha/Cargo.toml @@ -0,0 +1,24 @@ +[package] +name = "aprender-build-sha" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +description = "Build-script helper that stamps APR_GIT_SHA into every aprender binary's --version" +license = "MIT OR Apache-2.0" +repository = "https://github.com/paiml/aprender" +keywords = ["build", "version", "git", "sha", "aprender"] +categories = ["development-tools::build-utils"] +authors = ["PAIML Engineering"] + +# No dependencies, on purpose: this is a BUILD-dependency of every aprender +# binary package (#4219), so anything it pulled in would compile into every +# build script in the workspace. + +[lib] +name = "build_sha" + +[lints] +workspace = true + +[dev-dependencies] +tempfile = { workspace = true } diff --git a/crates/aprender-build-sha/README.md b/crates/aprender-build-sha/README.md new file mode 100644 index 0000000000..bb014fb3b9 --- /dev/null +++ b/crates/aprender-build-sha/README.md @@ -0,0 +1,36 @@ +# aprender-build-sha + +Build-script helper that stamps `APR_GIT_SHA` into every aprender binary, so +`--version` prints ` ()` — the shape `apr --version` has +printed since #597 (#4219). + +Part of the [Aprender](https://github.com/paiml/aprender) monorepo. + +## Use + +```toml +[build-dependencies] +aprender-build-sha = { workspace = true } +``` + +```rust +// build.rs +fn main() { + build_sha::emit(); +} +``` + +```rust +#[command(version = concat!(env!("CARGO_PKG_VERSION"), " (", env!("APR_GIT_SHA"), ")"))] +``` + +## Resolution order + +1. `APR_GIT_SHA_OVERRIDE` env var (CI/release hook) +2. `git rev-parse --short HEAD` +3. a committed `.git-sha` file (crates.io installs) +4. `v{CARGO_PKG_VERSION}+no-git` — never a bare "unknown" + +## Links + +- [Monorepo](https://github.com/paiml/aprender) diff --git a/crates/aprender-build-sha/src/emit_tests.rs b/crates/aprender-build-sha/src/emit_tests.rs new file mode 100644 index 0000000000..49806244dd --- /dev/null +++ b/crates/aprender-build-sha/src/emit_tests.rs @@ -0,0 +1,228 @@ +//! #4110 — a crates.io install of apr names its commit, never `+no-git`. Since #4219 every +//! workspace [[bin]] stamps its SHA through this crate, so the table runs against it. +//! +//! The 0.69.1 post-publish install printed `apr 0.69.1 (v0.69.1+no-git)` (0.68.1 did too) although +//! the published tarball carried the commit: `cargo package` writes `.cargo_vcs_info.json` into +//! every crate. These tests compile the SHIPPED `lib.rs` plus a `fn main() { emit() }` with rustc and run it the way cargo runs a +//! build script (CARGO_MANIFEST_DIR, CARGO_PKG_VERSION, cwd = the package dir) in a temp package +//! dir with no `.git` of ours above it. The vcs file is byte-exact from the real apr-cli-0.69.1 crate. +//! The release binaries printed `+no-git` too: binary-release.yml builds as root in a container over +//! the runner-owned checkout, and git refuses it ("dubious ownership"); row `foreign-owner`. +//! `planted_regressions_turn_red` re-runs the table on three planted copies of `lib.rs` — the vcs +//! rung removed (the pre-#4110 script), the vcs rung behind `git rev-parse`, and the safe.directory +//! retry removed — and requires each to fail, so the table cannot pass vacuously. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +const LIB_RS: &str = include_str!("lib.rs"); + +/// What a package's `build.rs` compiles to: this crate's code, entered through [`super::emit`]. +fn shipped() -> String { + format!("{LIB_RS}\nfn main() {{\n emit();\n}}\n") +} +const VCS_CALL: &str = " if let Some(sha) = vcs_info_sha() {"; + +/// Byte-exact `.cargo_vcs_info.json` of apr-cli-0.69.1 as downloaded from crates.io. +const VCS_REAL: &str = "{\n \"git\": {\n \"sha1\": \"d8a6df53a6c7104ef4eabfd2284908f93f1d6a3f\"\n },\n \"path_in_vcs\": \"crates/apr-cli\"\n}"; +const VCS_DIRTY: &str = "{\n \"git\": {\n \"sha1\": \"d8a6df53a6c7104ef4eabfd2284908f93f1d6a3f\",\n \"dirty\": true\n },\n \"path_in_vcs\": \"crates/apr-cli\"\n}"; +const VCS_BAD: &str = + "{\n \"git\": {\n \"sha1\": \"not-a-sha\"\n },\n \"path_in_vcs\": \"crates/apr-cli\"\n}"; + +fn compile(source: &str, dir: &Path, name: &str) -> PathBuf { + let src = dir.join(format!("{name}.rs")); + let bin = dir.join(name); + std::fs::write(&src, source).expect("write build script copy"); + let rustc = std::env::var("RUSTC").unwrap_or_else(|_| "rustc".to_string()); + let out = Command::new(rustc) + .args([ + "--edition", + "2021", + "--crate-name", + "build_script_build", + "-o", + ]) + .arg(&bin) + .arg(&src) + .output() + .expect("rustc must be runnable: this test compiles the shipped build-sha source"); + assert!( + out.status.success(), + "rustc rejected {name}: {}", + String::from_utf8_lossy(&out.stderr) + ); + bin +} + +fn git(dir: &Path, args: &[&str]) -> String { + let out = Command::new("git") + .args([ + "-c", + "core.hooksPath=/dev/null", + "-c", + "user.email=f@x", + "-c", + "user.name=f", + ]) + .args(args) + .current_dir(dir) + .output() + .expect("git must be runnable"); + assert!( + out.status.success(), + "git {args:?}: {}", + String::from_utf8_lossy(&out.stderr) + ); + String::from_utf8_lossy(&out.stdout).trim().to_string() +} + +/// A fresh package dir under `root`; the vcs file written when given; optionally its own repo. +fn package(root: &Path, name: &str, vcs: Option<&str>) -> PathBuf { + let dir = root.join(name); + std::fs::create_dir_all(&dir).expect("mkdir package"); + if let Some(v) = vcs { + std::fs::write(dir.join(".cargo_vcs_info.json"), v).expect("write vcs file"); + } + dir +} + +/// Runs the compiled build script like cargo does; returns the APR_GIT_SHA it emits. +/// `ceiling` stops git's upward search there (None: search freely, as a real install would). +fn run(bin: &Path, pkg: &Path, ceiling: Option<&Path>, env: &[(&str, &str)]) -> String { + let out_dir = pkg.join("target-out"); + std::fs::create_dir_all(&out_dir).expect("mkdir OUT_DIR"); + let mut cmd = Command::new(bin); + cmd.current_dir(pkg) + .env("CARGO_MANIFEST_DIR", pkg) + .env("OUT_DIR", &out_dir) + .env("CARGO_PKG_VERSION", "9.9.9") + .env_remove("APR_GIT_SHA_OVERRIDE") + .env_remove("GIT_DIR") + .env_remove("GIT_WORK_TREE") + .env( + "GIT_CEILING_DIRECTORIES", + ceiling.unwrap_or_else(|| Path::new("/nonexistent")), + ) + .envs(env.iter().copied()); + let out = cmd.output().expect("run build script"); + String::from_utf8_lossy(&out.stdout) + .lines() + .find_map(|l| l.strip_prefix("cargo:rustc-env=APR_GIT_SHA=")) + .unwrap_or("") + .to_string() +} + +/// The case table. Returns the names of the rows that landed wrong (empty = all as expected). +fn table(build_rs: &str, label: &str) -> Vec { + let tmp = tempfile::tempdir().expect("tempdir"); + let root = tmp.path(); + let bin = compile(build_rs, root, label); + let mut wrong = Vec::new(); + let mut expect = |row: &str, got: String, want: &str| { + if got != want { + wrong.push(format!("{row}: got {got:?}, want {want:?}")); + } + }; + let pkgs = root.join("pkgs"); + let p = package(&pkgs, "packaged", Some(VCS_REAL)); + expect("packaged", run(&bin, &p, Some(&pkgs), &[]), "d8a6df53a"); + // an unrelated repository ABOVE the unpacked crate, with a commit, and no ceiling between them + let foreign = package(&pkgs, "foreign", None); + git(&foreign, &["init", "-q"]); + git( + &foreign, + &["commit", "-q", "--allow-empty", "-m", "foreign"], + ); + let p = package(&foreign, "crate", Some(VCS_REAL)); + expect("foreign-repo", run(&bin, &p, None, &[]), "d8a6df53a"); + let p = package(&pkgs, "dirty", Some(VCS_DIRTY)); + expect("dirty", run(&bin, &p, Some(&pkgs), &[]), "d8a6df53a-dirty"); + let p = package(&pkgs, "malformed", Some(VCS_BAD)); + expect( + "malformed", + run(&bin, &p, Some(&pkgs), &[]), + "v9.9.9+no-git", + ); + let p = package(&pkgs, "no-vcs", None); + expect("no-vcs", run(&bin, &p, Some(&pkgs), &[]), "v9.9.9+no-git"); + let p = package(&pkgs, "override", Some(VCS_REAL)); + expect( + "override", + run( + &bin, + &p, + Some(&pkgs), + &[("APR_GIT_SHA_OVERRIDE", "release123")], + ), + "release123", + ); + // a dev checkout: no vcs file, its own repo -> `git rev-parse --short HEAD` (worktree freshness) + let dev = package(&pkgs, "dev", None); + git(&dev, &["init", "-q"]); + git(&dev, &["commit", "-q", "--allow-empty", "-m", "dev"]); + let head = git(&dev, &["rev-parse", "--short", "HEAD"]); + expect("dev-checkout", run(&bin, &dev, None, &[]), &head); + // the release lane: root in a container over the runner-owned checkout. git's own test knob + // makes it see another owner; the precondition proves plain git really refuses the repo here + let other_owner = [("GIT_TEST_ASSUME_DIFFERENT_OWNER", "1")]; + let refused = Command::new("git") + .args(["rev-parse", "HEAD"]) + .current_dir(&dev) + .envs(other_owner) + .output(); + assert!( + !refused.expect("git").status.success(), + "cannot simulate a checkout owned by another user: this git ignores GIT_TEST_ASSUME_DIFFERENT_OWNER" + ); + expect("foreign-owner", run(&bin, &dev, None, &other_owner), &head); + wrong +} + +#[test] +fn packaged_build_rs_names_its_commit() { + let wrong = table(&shipped(), "shipped"); + assert!( + wrong.is_empty(), + "shipped build-sha rows landed wrong: {wrong:#?}" + ); +} + +#[test] +fn planted_regressions_turn_red() { + let shipped = shipped(); + assert_eq!( + shipped.matches(VCS_CALL).count(), + 1, + "the vcs rung call site moved; re-anchor the plants" + ); + let no_rung = shipped.replace(VCS_CALL, " if let Some(sha) = None:: {"); + let git_first = shipped.replace( + VCS_CALL, + " if let Some(sha) = run_git(&[\"rev-parse\", \"--short\", \"HEAD\"]).or_else(vcs_info_sha) {", + ); + let w1 = table(&no_rung, "plant_no_rung"); + assert!( + w1.iter().any(|r| r.starts_with("packaged:")), + "no-vcs-rung plant survived: {w1:#?}" + ); + let retry = "run_git(&head).or_else(|| trusted_git_retry(&head))"; + assert_eq!( + shipped.matches(retry).count(), + 1, + "the git retry call site moved; re-anchor the plant" + ); + let w3 = table(&shipped.replace(retry, "run_git(&head)"), "plant_no_retry"); + assert!( + w3.iter().any(|r| r.starts_with("foreign-owner:")), + "no-retry plant survived: {w3:#?}" + ); + let w2 = table(&git_first, "plant_git_first"); + assert!( + w2.iter().any(|r| r.starts_with("foreign-repo:")), + "git-first plant survived: {w2:#?}" + ); + assert!( + !w2.iter().any(|r| r.starts_with("packaged:")), + "git-first plant should still pass packaged: {w2:#?}" + ); +} diff --git a/crates/aprender-build-sha/src/lib.rs b/crates/aprender-build-sha/src/lib.rs new file mode 100644 index 0000000000..d39a77015f --- /dev/null +++ b/crates/aprender-build-sha/src/lib.rs @@ -0,0 +1,256 @@ +//! Stamp `APR_GIT_SHA` into a binary at build time (#4219). +//! +//! Call [`emit`] from a package's `build.rs`; the binary then reads the value +//! with `env!("APR_GIT_SHA")` and prints ` ()` from +//! `--version`, the shape `apr` has printed since #597. +//! +//! Contract: apr-version-traceability-v1 F-VERSION-001..004 (paiml/aprender#597, +//! #1862). `APR_GIT_SHA` is resolved with a multi-source fallback hierarchy: +//! 1. `APR_GIT_SHA_OVERRIDE` env var (CI/release hook) +//! 2. `.cargo_vcs_info.json` `git.sha1` — written by `cargo package` into every published +//! tarball, so a crates.io install knows its commit (#4110). Ahead of `git rev-parse` +//! because an unpacked crate may sit inside SOME OTHER repository whose HEAD is not ours +//! 3. `git rev-parse --short HEAD` (dev builds from worktree or primary checkout; retried with +//! the enclosing checkout marked safe.directory when git refuses a checkout owned by another +//! user — binary-release.yml builds as root in a container over the runner's checkout) +//! 4. committed `.git-sha` file +//! 5. `v{CARGO_PKG_VERSION}+no-git` (informative fallback — never bare "unknown") +//! +//! This logic was moved out of `crates/apr-cli/build.rs`; the pure resolution +//! order is factored into [`resolve`] so it can be tested without running git, +//! and `emit_tests` compiles this file as a build script and runs it for real. + +/// Resolve `APR_GIT_SHA`, print it as `cargo:rustc-env`, and register the +/// rerun-if-changed triggers. Call once from `build.rs`. +pub fn emit() { + let sha = resolve_git_sha(); + println!("cargo:rustc-env=APR_GIT_SHA={sha}"); + + // Rerun build when HEAD moves. In a primary checkout `.git` is a directory; + // in a worktree `.git` is a file pointer to `/worktrees//`. + // Resolve the actual git directory(ies) via `git rev-parse` so both layouts + // are watched correctly (#1862). + register_git_rerun_triggers(); + + println!("cargo:rerun-if-env-changed=APR_GIT_SHA_OVERRIDE"); + println!("cargo:rerun-if-changed=.git-sha"); + println!("cargo:rerun-if-changed=.cargo_vcs_info.json"); +} + +/// The pure resolution order. Each argument is the raw result of one source, +/// `None` when that source was unavailable. The first non-empty (after trim) +/// source wins; `v{version}+no-git` is the fallback. +#[must_use] +pub fn resolve( + override_env: Option<&str>, + vcs_info: Option<&str>, + git_head: Option<&str>, + sha_file: Option<&str>, + version: Option<&str>, +) -> String { + // 1. Explicit override from CI/release + if let Some(s) = nonblank(override_env) { + return s.to_string(); + } + // 2. The commit `cargo package` recorded (crates.io / packaged builds) + if let Some(s) = nonblank(vcs_info) { + return s.to_string(); + } + // 3. Live git hash from worktree (works for both primary checkouts and worktrees) + if let Some(s) = nonblank(git_head) { + return s.to_string(); + } + // 4. Committed .git-sha fallback (updated by release script at publish time) + if let Some(s) = nonblank(sha_file) { + return s.to_string(); + } + // 5. Informative fallback — never bare "unknown" + format!("v{}+no-git", version.unwrap_or("0.0.0")) +} + +fn nonblank(s: Option<&str>) -> Option<&str> { + s.map(str::trim).filter(|s| !s.is_empty()) +} + +fn register_git_rerun_triggers() { + // Worktree-local HEAD: /HEAD. In a worktree this is the per-worktree + // HEAD pointer; in a primary checkout it's the same as /HEAD. + if let Some(git_dir) = run_git(&["rev-parse", "--git-dir"]) { + let head = format!("{git_dir}/HEAD"); + if std::path::Path::new(&head).exists() { + println!("cargo:rerun-if-changed={head}"); + } + } + // Shared refs live under /refs/heads/ regardless of layout. + if let Some(common_dir) = run_git(&["rev-parse", "--git-common-dir"]) { + let refs = format!("{common_dir}/refs/heads"); + if std::path::Path::new(&refs).exists() { + println!("cargo:rerun-if-changed={refs}"); + } + } +} + +fn run_git(args: &[&str]) -> Option { + let out = std::process::Command::new("git").args(args).output().ok()?; + if !out.status.success() { + return None; + } + let s = String::from_utf8_lossy(&out.stdout).trim().to_string(); + if s.is_empty() { + None + } else { + Some(s) + } +} + +fn resolve_git_sha() -> String { + // Sources are consulted lazily and in order, as the original apr-cli + // build.rs did: git is not run when an earlier source answers, and + // `.git-sha` is not read when git answers. + let override_env = std::env::var("APR_GIT_SHA_OVERRIDE").ok(); + if let Some(s) = nonblank(override_env.as_deref()) { + return s.to_string(); + } + if let Some(sha) = vcs_info_sha() { + return sha; + } + // `trusted_git_retry`: the release binaries are built as root in a container over the + // runner-owned checkout, where git refuses the repo as "dubious ownership" (#4110) + let head = ["rev-parse", "--short", "HEAD"]; + let git_head = run_git(&head).or_else(|| trusted_git_retry(&head)); + let sha_file = match git_head { + Some(_) => None, + None => std::fs::read_to_string(".git-sha").ok(), + }; + let version = std::env::var("CARGO_PKG_VERSION").ok(); + resolve( + None, + None, + git_head.as_deref(), + sha_file.as_deref(), + version.as_deref(), + ) +} + +/// `git.sha1` from `.cargo_vcs_info.json`, shortened to 9 hex digits (what `git rev-parse +/// --short` prints in this repository), with `-dirty` when cargo packaged a dirty tree. +/// Parsed by hand: this crate has no dependencies, and the file is cargo's own fixed shape. +fn vcs_info_sha() -> Option { + let dir = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".to_string()); + let text = std::fs::read_to_string(format!("{dir}/.cargo_vcs_info.json")).ok()?; + let after = &text[text.find("\"sha1\"")? + "\"sha1\"".len()..]; + let open = after.find('"')?; + let value = &after[open + 1..]; + let sha = &value[..value.find('"')?]; + if sha.len() != 40 || !sha.bytes().all(|b| b.is_ascii_hexdigit()) { + return None; + } + let dirty = text.contains("\"dirty\": true") || text.contains("\"dirty\":true"); + Some(format!( + "{}{}", + &sha[..9], + if dirty { "-dirty" } else { "" } + )) +} + +/// `git ` again, with the checkout that encloses this package marked `safe.directory`. +/// +/// git refuses a repository owned by another user ("detected dubious ownership"), and the release +/// lane builds as root in a container over the runner-owned checkout, so every published binary +/// printed `+no-git` (#4110). The exception goes in a private GLOBAL config under OUT_DIR: +/// git ignores `safe.directory` given by `-c` or GIT_CONFIG_* before 2.39 (measured, 2.34.1). +/// Only the checkout found by walking up from CARGO_MANIFEST_DIR is trusted, never `*`, and +/// `rev-parse` runs no hooks. This build is already compiling that checkout's code. +fn trusted_git_retry(args: &[&str]) -> Option { + let manifest = std::env::var("CARGO_MANIFEST_DIR").ok()?; + let top = std::path::Path::new(&manifest) + .ancestors() + .find(|d| d.join(".git").exists())?; + let config = + std::path::Path::new(&std::env::var("OUT_DIR").ok()?).join("apr-safe-directory.gitconfig"); + std::fs::write( + &config, + format!("[safe]\n\tdirectory = {}\n", top.display()), + ) + .ok()?; + let out = std::process::Command::new("git") + .args(args) + .env("GIT_CONFIG_GLOBAL", &config) + .output() + .ok()?; + let s = String::from_utf8_lossy(&out.stdout).trim().to_string(); + (out.status.success() && !s.is_empty()).then_some(s) +} + +#[cfg(test)] +mod emit_tests; + +#[cfg(test)] +mod tests { + use super::resolve; + + #[test] + fn override_wins_over_everything() { + assert_eq!( + resolve( + Some(" abc123456 \n"), + None, + Some("def456789"), + Some("0123456789"), + Some("1.2.3") + ), + "abc123456" + ); + } + + #[test] + fn blank_override_falls_through_to_git() { + assert_eq!( + resolve( + Some(" "), + None, + Some("def456789"), + Some("0123456789"), + Some("1.2.3") + ), + "def456789" + ); + assert_eq!( + resolve(None, None, Some("def456789"), None, None), + "def456789" + ); + } + + #[test] + fn packaged_commit_wins_over_git() { + assert_eq!( + resolve(None, Some("d8a6df53a"), Some("def456789"), None, None), + "d8a6df53a" + ); + assert_eq!( + resolve(Some("rel123"), Some("d8a6df53a"), None, None, None), + "rel123" + ); + } + + #[test] + fn sha_file_used_only_without_git() { + assert_eq!( + resolve(None, None, None, Some("0123456789\n"), Some("1.2.3")), + "0123456789" + ); + } + + #[test] + fn no_git_fallback_names_the_version_never_unknown() { + assert_eq!( + resolve(None, None, None, None, Some("0.69.0")), + "v0.69.0+no-git" + ); + assert_eq!( + resolve(Some(""), None, None, Some(" \n"), Some("0.69.0")), + "v0.69.0+no-git" + ); + assert_eq!(resolve(None, None, None, None, None), "v0.0.0+no-git"); + } +} diff --git a/crates/aprender-cbtop/Cargo.toml b/crates/aprender-cbtop/Cargo.toml index ffb6420647..1a97f9cb27 100644 --- a/crates/aprender-cbtop/Cargo.toml +++ b/crates/aprender-cbtop/Cargo.toml @@ -55,6 +55,9 @@ dirs = "5.0" [target.'cfg(unix)'.dependencies] libc = "0.2" +[build-dependencies] +aprender-build-sha = { workspace = true } # #4219: APR_GIT_SHA for --version + [dev-dependencies] proptest = "1" tempfile = "3" diff --git a/crates/aprender-cbtop/build.rs b/crates/aprender-cbtop/build.rs new file mode 100644 index 0000000000..961a8293c3 --- /dev/null +++ b/crates/aprender-cbtop/build.rs @@ -0,0 +1,6 @@ +// #4219: stamp APR_GIT_SHA so `--version` prints ` ()`, +// the shape `apr --version` has carried since #597. Resolution order and +// rerun triggers live in the shared `aprender-build-sha` crate. +fn main() { + build_sha::emit(); +} diff --git a/crates/aprender-cbtop/src/main.rs b/crates/aprender-cbtop/src/main.rs index fe1ee1f90b..cd38813b7b 100644 --- a/crates/aprender-cbtop/src/main.rs +++ b/crates/aprender-cbtop/src/main.rs @@ -18,7 +18,7 @@ use clap::{Parser, Subcommand}; #[derive(Parser, Debug)] #[command(name = "cbtop")] #[command(author = "Trueno Engineering")] -#[command(version)] +#[command(version = concat!(env!("CARGO_PKG_VERSION"), " (", env!("APR_GIT_SHA"), ")"))] #[command(about = "Real-time load testing and hardware monitoring TUI", long_about = None)] struct Cli { /// Subcommand (bench for headless benchmarking) diff --git a/crates/aprender-cgp/Cargo.toml b/crates/aprender-cgp/Cargo.toml index 324a489aa8..8815bc10a3 100644 --- a/crates/aprender-cgp/Cargo.toml +++ b/crates/aprender-cgp/Cargo.toml @@ -66,6 +66,9 @@ cuda = ["dep:trueno-gpu"] # at release time. Not in CI's default set, and not in the nightly full surface. timing-gate = [] +[build-dependencies] +aprender-build-sha = { workspace = true } # #4219: APR_GIT_SHA for --version + [dev-dependencies] proptest = "1.9" tempfile = "3" diff --git a/crates/aprender-cgp/build.rs b/crates/aprender-cgp/build.rs new file mode 100644 index 0000000000..961a8293c3 --- /dev/null +++ b/crates/aprender-cgp/build.rs @@ -0,0 +1,6 @@ +// #4219: stamp APR_GIT_SHA so `--version` prints ` ()`, +// the shape `apr --version` has carried since #597. Resolution order and +// rerun triggers live in the shared `aprender-build-sha` crate. +fn main() { + build_sha::emit(); +} diff --git a/crates/aprender-cgp/src/cli.rs b/crates/aprender-cgp/src/cli.rs index 66ac457ab1..3af89083bc 100644 --- a/crates/aprender-cgp/src/cli.rs +++ b/crates/aprender-cgp/src/cli.rs @@ -24,7 +24,7 @@ pub const CGP_BACKEND_VALUES: [&str; 7] = /// Profiles scalar, SIMD (SSE2/AVX2/AVX-512/NEON/WASM SIMD128), /// wgpu (Vulkan/Metal/DX12/WebGPU), and CUDA workloads. #[derive(Parser, Clone, Debug)] -#[command(name = "cgp", version, about, long_about = None)] +#[command(name = "cgp", version = concat!(env!("CARGO_PKG_VERSION"), " (", env!("APR_GIT_SHA"), ")"), about, long_about = None)] pub struct Cli { /// Output JSON instead of human-readable text #[arg(long, global = true)] diff --git a/crates/aprender-common/Cargo.toml b/crates/aprender-common/Cargo.toml index 1861e9cd0b..6bd5a61a52 100644 --- a/crates/aprender-common/Cargo.toml +++ b/crates/aprender-common/Cargo.toml @@ -12,7 +12,14 @@ categories = ["command-line-utilities", "text-processing"] [lib] name = "batuta_common" +[features] +default = [] +# #3745 S1: typed CLI argument roles (`batuta_common::cli_roles`). Optional so the +# default build of this crate stays clap-free. +cli-roles = ["dep:clap"] + [dependencies] +clap = { workspace = true, optional = true } lz4_flex = { version = "0.11", default-features = false, features = ["safe-encode", "safe-decode"] } zstd = { version = "0.13", default-features = false } diff --git a/crates/aprender-common/src/cli_roles.rs b/crates/aprender-common/src/cli_roles.rs new file mode 100644 index 0000000000..cd0da3c474 --- /dev/null +++ b/crates/aprender-common/src/cli_roles.rs @@ -0,0 +1,732 @@ +//! Typed CLI argument roles (#3745 S1, PMAT-3749). +//! +//! A release cell can only be derived for an argument whose ROLE the gate can +//! see. Before these types, the role of an `apr` argument was knowable only +//! from its name: `apr run SOURCE`, `--prompt` and `-i FILE` were a `String`, a +//! `String` and a `PathBuf`, the same types as `--name`, `--tag` and +//! `--output`. Every release list was therefore typed by hand, and a verb, flag +//! or input shape missing from that list was invisible to the gate. That is how +//! `apr run --prompt P --chat` shipped double-templated (#3743) while +//! `-i file --chat` was covered. +//! +//! Each type here is a transparent newtype over `PathBuf` or `String` whose clap +//! value parser is the SAME parser clap uses for the raw type +//! (`PathBufValueParser` / `StringValueParser`), mapped into the newtype. So a +//! migrated argument accepts exactly the values it accepted before. The only +//! thing that changes is the parser's output TYPE, and that is what `apr surface` +//! reads, through `ValueParser::type_id()`, to decide the role. The role is +//! declared by how the argument is BUILT; its name never enters into it. +//! +//! The types live here rather than in `apr-cli` because a marker compared by +//! `TypeId` needs ONE defining crate, and some of `apr`'s surface is declared by +//! crates that do not depend on `apr-cli` (`apr pv …` from +//! `aprender-contracts-cli`, `apr rag …` from `aprender-rag-cli`). + +use std::any::TypeId; +use std::ffi::OsStr; +use std::fmt; +use std::ops::Deref; +use std::path::{Path, PathBuf}; + +use clap::builder::{MapValueParser, PathBufValueParser, StringValueParser, TypedValueParser}; + +/// The role an argument plays in a release cell (#3745). +/// +/// `Mode` and `Backend` are not listed in [`MARKERS`]. Mode is structural (a +/// flag action, or a finite value set), and backend is membership in the shared +/// `BackendArg` group; `apr surface` decides both. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub enum Role { + /// A model: a local file, or a reference `apr` resolves to one. + Model, + /// Text the model is prompted with. + Prompt, + /// A file the command consumes as input data (audio, text, a dataset). + InputFile, + /// The compute backend override. + Backend, + /// A generation sampling control: seed, temperature, top-k, top-p, + /// repeat penalty (S1 v1.1, for S2.5's flag-effect sampling controls). + Sampling, + /// A flag, or a value drawn from a finite set. + Mode, + /// Declared, and none of the above: outputs, directories, configs, names. + Other, + /// A free-form value no role type declares. Only a foreign subtree may + /// carry one; the marker guard is RED on it anywhere else. + Unknown, +} + +impl Role { + /// The spelling used in the `apr-cli-surface` JSON. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::Model => "model", + Self::Prompt => "prompt", + Self::InputFile => "input-file", + Self::Backend => "backend", + Self::Sampling => "sampling", + Self::Mode => "mode", + Self::Other => "other", + Self::Unknown => "unknown", + } + } +} + +/// What a marker's underlying value is: a filesystem path, or free text. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Carrier { + /// Backed by `PathBuf`. + Path, + /// Backed by `String`. + Text, +} + +/// One role type: the `TypeId` its value parser yields, and what it declares. +#[derive(Debug, Clone, Copy)] +pub struct Marker { + /// The type's name as written in code, e.g. `ModelPath`. + pub name: &'static str, + /// The role this type declares. + pub role: Role, + /// Path- or text-backed. + pub carrier: Carrier, + type_id: fn() -> TypeId, +} + +impl Marker { + /// The `TypeId` of the marker type, which is the `TypeId` its clap value + /// parser reports. + #[must_use] + pub fn type_id(&self) -> TypeId { + (self.type_id)() + } +} + +macro_rules! path_role { + ($(#[$doc:meta])* $name:ident) => { + $(#[$doc])* + #[derive(Clone, Default, PartialEq, Eq, Hash, PartialOrd, Ord)] + pub struct $name(PathBuf); + + // `Debug` is the inner value's, so `{:?}` of a migrated field prints + // `"m.gguf"` exactly as the raw `PathBuf` did, never `ModelPath("m.gguf")`. + impl fmt::Debug for $name { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&self.0, f) + } + } + + impl $name { + /// Wrap a path. + #[must_use] + pub fn new(path: impl Into) -> Self { + Self(path.into()) + } + /// The path, borrowed. + #[must_use] + pub fn as_path(&self) -> &Path { + &self.0 + } + /// The path, owned. + #[must_use] + pub fn into_path_buf(self) -> PathBuf { + self.0 + } + } + + // `Path`, not `PathBuf`: the same target `PathBuf` derefs to, so + // `Option::as_deref()` is `Option<&Path>` exactly as + // `Option::as_deref()` was, and a migrated field reads the same. + impl Deref for $name { + type Target = Path; + fn deref(&self) -> &Path { + &self.0 + } + } + impl AsRef for $name { + fn as_ref(&self) -> &Path { + &self.0 + } + } + impl AsRef for $name { + fn as_ref(&self) -> &OsStr { + self.0.as_os_str() + } + } + impl From for $name { + fn from(p: PathBuf) -> Self { + Self(p) + } + } + impl From<&Path> for $name { + fn from(p: &Path) -> Self { + Self(p.to_path_buf()) + } + } + impl From<&str> for $name { + fn from(p: &str) -> Self { + Self(PathBuf::from(p)) + } + } + impl From for $name { + fn from(p: String) -> Self { + Self(PathBuf::from(p)) + } + } + impl From<$name> for PathBuf { + fn from(p: $name) -> PathBuf { + p.0 + } + } + impl PartialEq for $name { + fn eq(&self, other: &PathBuf) -> bool { + &self.0 == other + } + } + impl PartialEq for $name { + fn eq(&self, other: &Path) -> bool { + self.0 == other + } + } + impl clap::builder::ValueParserFactory for $name { + type Parser = MapValueParser $name>; + fn value_parser() -> Self::Parser { + PathBufValueParser::new().map($name as fn(PathBuf) -> $name) + } + } + }; +} + +macro_rules! text_role { + ($(#[$doc:meta])* $name:ident) => { + $(#[$doc])* + #[derive(Clone, Default, PartialEq, Eq, Hash, PartialOrd, Ord)] + pub struct $name(String); + + // `Debug` is the inner value's: `{:?}` prints `"text"`, as `String` did. + impl fmt::Debug for $name { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + fmt::Debug::fmt(&self.0, f) + } + } + + impl $name { + /// Wrap a string. + #[must_use] + pub fn new(s: impl Into) -> Self { + Self(s.into()) + } + /// The text, borrowed. + #[must_use] + pub fn as_str(&self) -> &str { + &self.0 + } + /// The text as a `&String`, for callees that take one. + #[must_use] + pub fn as_string(&self) -> &String { + &self.0 + } + /// The text, owned. + #[must_use] + pub fn into_string(self) -> String { + self.0 + } + } + + // `str`, not `String`: the same target `String` derefs to, so + // `Option::as_deref()` is `Option<&str>` exactly as + // `Option::as_deref()` was. + impl Deref for $name { + type Target = str; + fn deref(&self) -> &str { + &self.0 + } + } + impl AsRef for $name { + fn as_ref(&self) -> &str { + &self.0 + } + } + impl AsRef for $name { + fn as_ref(&self) -> &Path { + Path::new(&self.0) + } + } + impl AsRef for $name { + fn as_ref(&self) -> &OsStr { + OsStr::new(&self.0) + } + } + impl fmt::Display for $name { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(&self.0) + } + } + impl From for $name { + fn from(s: String) -> Self { + Self(s) + } + } + impl From<&str> for $name { + fn from(s: &str) -> Self { + Self(s.to_string()) + } + } + impl From<$name> for String { + fn from(s: $name) -> String { + s.0 + } + } + impl PartialEq for $name { + fn eq(&self, other: &str) -> bool { + self.0 == other + } + } + impl PartialEq<&str> for $name { + fn eq(&self, other: &&str) -> bool { + self.0 == *other + } + } + impl PartialEq for $name { + fn eq(&self, other: &String) -> bool { + &self.0 == other + } + } + impl clap::builder::ValueParserFactory for $name { + type Parser = MapValueParser $name>; + fn value_parser() -> Self::Parser { + StringValueParser::new().map($name as fn(String) -> $name) + } + } + }; +} + +path_role!( + /// A local model file (`.apr`, `.gguf`, `.safetensors`, …). + ModelPath +); +path_role!( + /// A file the command reads as input data, not as a model. + InputFile +); +path_role!( + /// A file the command writes. + OutputPath +); +path_role!( + /// A directory: a workspace, a cache, a corpus root, an output tree. + DirPath +); +path_role!( + /// A configuration, manifest, contract, schema or receipt the command + /// reads to decide what to do, as distinct from the data it operates on. + ConfigPath +); +text_role!( + /// A model reference: a local path, `hf://org/repo`, a URL, or a cache + /// name, which `apr` resolves to a model file. + ModelRef +); +text_role!( + /// Text the model is prompted with. + PromptText +); +text_role!( + /// Any other free text: a name, id, tag, URL, pattern or list. + FreeText +); +text_role!( + /// Text a model ENCODES or SCORES rather than generates from: `apr embed + /// --text`, `apr rerank --query/--passage`, `apr eval --text` (perplexity). + /// Role `other`, so the command does not read as a generator: `generates` + /// drives the thinking × context-rung cells, which mean nothing for an + /// encoder or a scorer (aprender-97 on #3745, v1.1). + EncodeText +); + +/// Command-level marker: this command GENERATES text from prompts that do not +/// arrive through a `PromptText` argument (#3745 S1, schema v1.1). +/// +/// `apr serve run` takes its prompts over HTTP, `apr chat` reads them from +/// stdin, and `apr mcp` receives them as JSON-RPC tool calls. None of them has a +/// prompt ARGUMENT, so a rule that reads "generates" off the prompt role alone +/// would derive no prompt-shape cells for the very verb whose 0.69.0 leak +/// (#3571) started this. The marker is attached to the subcommand itself, with +/// no argument and no field: +/// +/// ```ignore +/// #[command(group(batuta_common::cli_roles::ServesGeneration::group()))] +/// Run { /* … */ }, +/// ``` +/// +/// `apr surface` reports `generates: true` for a command that has a +/// `PromptText` argument OR carries this group, and it matches the group by +/// [`ServesGeneration::ID`], never by the command's name. +#[derive(Debug, Clone, Copy)] +pub struct ServesGeneration; + +impl ServesGeneration { + /// The group id the marker is recognised by. + pub const ID: &'static str = "batuta_common::cli_roles::ServesGeneration"; + + /// The argument-less group that marks a command as a generator. It names + /// no argument, so it changes neither parsing nor `--help`. + #[must_use] + pub fn group() -> clap::ArgGroup { + clap::ArgGroup::new(Self::ID).multiple(true) + } + + /// True iff `cmd` carries the marker. + #[must_use] + pub fn is_on(cmd: &clap::Command) -> bool { + cmd.get_groups().any(|g| g.get_id() == Self::ID) + } +} + +/// Which generation sampling control an argument is (#3745 S1, v1.1). S2 +/// derives its sampling cells per kind (sampled ≠ greedy, same seed twice is +/// byte-identical, seed A ≠ seed B, top-k 1 / T 0 is byte-greedy), so the kind +/// is declared, not read off the argument's name. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub enum SamplingKind { + /// RNG seed of the sampler. + Seed, + /// Softmax temperature. + Temperature, + /// Top-k cut-off. + TopK, + /// Nucleus (top-p) threshold. + TopP, + /// Min-p threshold. + MinP, + /// Repetition penalty. + RepeatPenalty, + /// Window the repetition penalty looks back over. + RepeatLastN, +} + +impl SamplingKind { + /// Every kind. + pub const ALL: [SamplingKind; 7] = [ + Self::Seed, + Self::Temperature, + Self::TopK, + Self::TopP, + Self::MinP, + Self::RepeatPenalty, + Self::RepeatLastN, + ]; + + /// The spelling used in the `apr-cli-surface` JSON (`sampling_kind`). + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::Seed => "seed", + Self::Temperature => "temperature", + Self::TopK => "top_k", + Self::TopP => "top_p", + Self::MinP => "min_p", + Self::RepeatPenalty => "repeat_penalty", + Self::RepeatLastN => "repeat_last_n", + } + } + + /// The group id an argument of this kind joins: `#[arg(group = SamplingKind::Temperature.id())]`. + #[must_use] + pub const fn id(self) -> &'static str { + match self { + Self::Seed => "batuta_common::cli_roles::SamplingArg::seed", + Self::Temperature => "batuta_common::cli_roles::SamplingArg::temperature", + Self::TopK => "batuta_common::cli_roles::SamplingArg::top_k", + Self::TopP => "batuta_common::cli_roles::SamplingArg::top_p", + Self::MinP => "batuta_common::cli_roles::SamplingArg::min_p", + Self::RepeatPenalty => "batuta_common::cli_roles::SamplingArg::repeat_penalty", + Self::RepeatLastN => "batuta_common::cli_roles::SamplingArg::repeat_last_n", + } + } +} + +/// Argument-level marker for a generation SAMPLING control (#3745 S1, v1.1). +/// +/// These are numbers, so a newtype would change every field's type and every +/// call site that reads it. Instead the argument joins the argument-less group +/// of its [`SamplingKind`], and the command declares those groups once, each +/// `multiple(true)`, so nothing about parsing changes: +/// +/// ```ignore +/// #[command(groups(batuta_common::cli_roles::SamplingArg::groups()))] +/// Run { +/// #[arg(long, default_value = "0.0", +/// group = batuta_common::cli_roles::SamplingKind::Temperature.id())] +/// temperature: f32, +/// } +/// ``` +/// +/// `apr surface` reports role `sampling` and `sampling_kind` for a member of +/// one of these groups, matched by id, never by the argument's name. +#[derive(Debug, Clone, Copy)] +pub struct SamplingArg; + +impl SamplingArg { + /// One argument-less `multiple(true)` group per [`SamplingKind`], for + /// `#[command(groups(SamplingArg::groups()))]`. + #[must_use] + pub fn groups() -> Vec { + SamplingKind::ALL + .iter() + .map(|k| clap::ArgGroup::new(k.id()).multiple(true)) + .collect() + } + + /// The sampling kind of `arg` on `built`, if it has one. `built` must be a + /// BUILT command: clap merges each argument's `group = …` into the + /// command's groups only when the command is built. + #[must_use] + pub fn kind_of(built: &clap::Command, arg: &clap::Arg) -> Option { + SamplingKind::ALL.into_iter().find(|k| { + built + .get_groups() + .any(|g| g.get_id() == k.id() && g.get_args().any(|a| a == arg.get_id())) + }) + } +} + +/// Owned `String`s from a slice of text role values, for callees that take +/// `&[String]`. +#[must_use] +pub fn strings>(values: &[T]) -> Vec { + values.iter().map(|v| v.as_ref().to_string()).collect() +} + +/// Owned `PathBuf`s from a slice of path role values, for callees that take +/// `&[PathBuf]`. +#[must_use] +pub fn path_bufs>(values: &[T]) -> Vec { + values.iter().map(|v| v.as_ref().to_path_buf()).collect() +} + +/// Every role type, in one table. `apr surface` reads roles through it, and the +/// marker guard refuses a free-form argument whose parser yields none of these. +pub const MARKERS: &[Marker] = &[ + Marker { + name: "ModelPath", + role: Role::Model, + carrier: Carrier::Path, + type_id: TypeId::of::, + }, + Marker { + name: "ModelRef", + role: Role::Model, + carrier: Carrier::Text, + type_id: TypeId::of::, + }, + Marker { + name: "PromptText", + role: Role::Prompt, + carrier: Carrier::Text, + type_id: TypeId::of::, + }, + Marker { + name: "InputFile", + role: Role::InputFile, + carrier: Carrier::Path, + type_id: TypeId::of::, + }, + Marker { + name: "OutputPath", + role: Role::Other, + carrier: Carrier::Path, + type_id: TypeId::of::, + }, + Marker { + name: "DirPath", + role: Role::Other, + carrier: Carrier::Path, + type_id: TypeId::of::, + }, + Marker { + name: "ConfigPath", + role: Role::Other, + carrier: Carrier::Path, + type_id: TypeId::of::, + }, + Marker { + name: "FreeText", + role: Role::Other, + carrier: Carrier::Text, + type_id: TypeId::of::, + }, + Marker { + name: "EncodeText", + role: Role::Other, + carrier: Carrier::Text, + type_id: TypeId::of::, + }, +]; + +/// The marker whose type a value parser yields, if any. +#[must_use] +pub fn marker_for(type_id: TypeId) -> Option<&'static Marker> { + MARKERS.iter().find(|m| m.type_id() == type_id) +} + +#[cfg(test)] +mod tests { + use super::*; + use clap::{Arg, Command}; + + fn parser_type() -> TypeId + where + T::Parser: TypedValueParser + Send + Sync + 'static, + ::Value: Send + Sync + Clone + 'static, + { + let vp: clap::builder::ValueParser = T::value_parser().into(); + let arg = Arg::new("x").value_parser(vp); + // `AnyValueId` is clap's; it compares equal to a std `TypeId`. + let id = arg.get_value_parser().type_id(); + MARKERS + .iter() + .map(Marker::type_id) + .chain([TypeId::of::(), TypeId::of::()]) + .find(|t| id == *t) + .expect("value parser yields a known type") + } + + #[test] + fn every_marker_parser_reports_its_own_type_id() { + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + assert_eq!(parser_type::(), TypeId::of::()); + } + + #[test] + fn marker_table_is_one_to_one_with_types() { + let mut ids: Vec = MARKERS.iter().map(Marker::type_id).collect(); + ids.sort(); + ids.dedup(); + assert_eq!(ids.len(), MARKERS.len(), "two markers share a TypeId"); + assert!( + marker_for(TypeId::of::()).is_none(), + "raw PathBuf must not be a marker" + ); + assert!( + marker_for(TypeId::of::()).is_none(), + "raw String must not be a marker" + ); + assert_eq!( + marker_for(TypeId::of::()).map(|m| m.role), + Some(Role::Model) + ); + assert_eq!( + marker_for(TypeId::of::()).map(|m| m.role), + Some(Role::Prompt) + ); + } + + /// A migrated argument must accept exactly what the raw type accepted: the + /// marker parser is the raw parser, mapped. An empty path is refused by + /// both, a normal value is accepted by both. + #[test] + fn marker_parsers_accept_what_the_raw_parsers_accept() { + let raw = Command::new("t").arg(Arg::new("p").value_parser(clap::value_parser!(PathBuf))); + let typed = + Command::new("t").arg(Arg::new("p").value_parser(clap::value_parser!(ModelPath))); + for argv in [vec!["t", "m.gguf"], vec!["t", ""], vec!["t", "-"]] { + let a = raw.clone().try_get_matches_from(&argv).is_ok(); + let b = typed.clone().try_get_matches_from(&argv).is_ok(); + assert_eq!(a, b, "PathBuf vs ModelPath disagree on {argv:?}"); + } + let m = typed.try_get_matches_from(["t", "m.gguf"]).expect("parses"); + assert_eq!( + m.get_one::("p").map(ModelPath::as_path), + Some(Path::new("m.gguf")) + ); + + let raw = Command::new("t").arg(Arg::new("s").value_parser(clap::value_parser!(String))); + let typed = + Command::new("t").arg(Arg::new("s").value_parser(clap::value_parser!(PromptText))); + for argv in [vec!["t", "What is 2+2?"], vec!["t", ""]] { + let a = raw.clone().try_get_matches_from(&argv).is_ok(); + let b = typed.clone().try_get_matches_from(&argv).is_ok(); + assert_eq!(a, b, "String vs PromptText disagree on {argv:?}"); + } + } + + /// The generation marker names no argument, so it changes neither what + /// parses nor what `--help` prints, and it is found by id alone. + #[test] + fn serves_generation_marks_a_command_without_changing_it() { + let plain = Command::new("t").arg(Arg::new("port").long("port")); + let marked = plain.clone().group(ServesGeneration::group()); + assert!(!ServesGeneration::is_on(&plain)); + assert!(ServesGeneration::is_on(&marked)); + for argv in [vec!["t"], vec!["t", "--port", "8080"], vec!["t", "--nope"]] { + let a = plain.clone().try_get_matches_from(&argv).is_ok(); + let b = marked.clone().try_get_matches_from(&argv).is_ok(); + assert_eq!(a, b, "the marker changed parsing of {argv:?}"); + } + let help = |c: &Command| c.clone().render_long_help().to_string(); + assert_eq!(help(&plain), help(&marked), "the marker changed --help"); + // A look-alike group spelled with the type's short name is not the marker. + let lookalike = plain.group(clap::ArgGroup::new("ServesGeneration").multiple(true)); + assert!(!ServesGeneration::is_on(&lookalike)); + } + + /// A migrated field must print the same under `{:?}` as the raw type did: + /// error messages and tests format arguments that way. + #[test] + fn debug_output_is_the_raw_types() { + assert_eq!( + format!("{:?}", ModelPath::from("m.gguf")), + format!("{:?}", PathBuf::from("m.gguf")) + ); + assert_eq!( + format!("{:?}", Some(OutputPath::from("o"))), + format!("{:?}", Some(PathBuf::from("o"))) + ); + assert_eq!( + format!("{:?}", PromptText::from("hi")), + format!("{:?}", String::from("hi")) + ); + assert_eq!( + format!("{:?}", vec![FreeText::from("a")]), + format!("{:?}", vec![String::from("a")]) + ); + } + + #[test] + fn derive_picks_the_marker_parser_for_a_marker_field() { + #[derive(clap::Parser)] + struct T { + model: ModelPath, + #[arg(long)] + prompt: Option, + #[arg(long)] + inputs: Vec, + } + use clap::{CommandFactory, Parser}; + let cmd = T::command(); + let id = |name: &str| { + cmd.get_arguments() + .find(|a| a.get_id() == name) + .map(|a| a.get_value_parser().type_id()) + .expect("arg exists") + }; + assert!(id("model") == TypeId::of::()); + assert!(id("prompt") == TypeId::of::()); + assert!(id("inputs") == TypeId::of::()); + let t = T::try_parse_from(["t", "a.apr", "--prompt", "hi", "--inputs", "x.wav"]) + .expect("parses"); + assert_eq!(t.model.as_path(), Path::new("a.apr")); + assert_eq!(t.prompt.as_deref(), Some("hi")); + assert_eq!(t.inputs.len(), 1); + } +} diff --git a/crates/aprender-common/src/lib.rs b/crates/aprender-common/src/lib.rs index 7ee0cb44c6..4128ca843a 100644 --- a/crates/aprender-common/src/lib.rs +++ b/crates/aprender-common/src/lib.rs @@ -8,11 +8,14 @@ //! - [`fmt`] — Byte, percentage, duration, and number formatting //! - [`sys`] — System information (cgroup detection, CPU info) //! - [`display`] — Column alignment, truncation, and builder traits +//! - `cli_roles` — typed CLI argument roles (feature `cli-roles`, #3745) // `.unwrap()` is banned in production via workspace `.clippy.toml`, but is // idiomatic in `#[cfg(test)]` assertions. Allow it only under `cfg(test)`. #![cfg_attr(test, allow(clippy::disallowed_methods))] +#[cfg(feature = "cli-roles")] +pub mod cli_roles; pub mod compression; pub mod display; pub mod fmt; diff --git a/crates/aprender-compute-xtask/Cargo.toml b/crates/aprender-compute-xtask/Cargo.toml index 1b14bfa5fa..6bb09f26d7 100644 --- a/crates/aprender-compute-xtask/Cargo.toml +++ b/crates/aprender-compute-xtask/Cargo.toml @@ -12,6 +12,9 @@ walkdir = "2.4" regex = "1.10" bashrs = "6.35" +[build-dependencies] +aprender-build-sha = { workspace = true } # #4219: APR_GIT_SHA for --version + [dev-dependencies] tempfile = "3.8" serial_test = "3.0" diff --git a/crates/aprender-compute-xtask/build.rs b/crates/aprender-compute-xtask/build.rs new file mode 100644 index 0000000000..f60fc6f23d --- /dev/null +++ b/crates/aprender-compute-xtask/build.rs @@ -0,0 +1,6 @@ +//! #4219: stamp `APR_GIT_SHA` so `--version` prints ` ()`, +//! the shape `apr --version` has carried since #597. Resolution order and +//! rerun triggers live in the shared `aprender-build-sha` crate. +fn main() { + build_sha::emit(); +} diff --git a/crates/aprender-compute-xtask/src/main.rs b/crates/aprender-compute-xtask/src/main.rs index 2e3dcea74c..713d6f4db0 100644 --- a/crates/aprender-compute-xtask/src/main.rs +++ b/crates/aprender-compute-xtask/src/main.rs @@ -24,7 +24,7 @@ use std::process::ExitCode; /// Trueno development task runner. #[derive(Debug, Parser)] #[command( - version, + version = concat!(env!("CARGO_PKG_VERSION"), " (", env!("APR_GIT_SHA"), ")"), about = "Trueno development task runner (xtask pattern)", long_about = "Trueno development task runner (xtask pattern).\n\n\ Normally invoked as `cargo xtask `." diff --git a/crates/aprender-compute/Cargo.toml b/crates/aprender-compute/Cargo.toml index f4be32328d..baf4510093 100644 --- a/crates/aprender-compute/Cargo.toml +++ b/crates/aprender-compute/Cargo.toml @@ -84,6 +84,7 @@ web-sys = { version = "0.3", optional = true, features = ["console"] } hostname = "0.4" [build-dependencies] +provable-contracts = { workspace = true } # #4369: build_helper::env_key, the one CONTRACT_* key builder serde = { version = "1", features = ["derive"] } serde_yaml_ng = "0.10" diff --git a/crates/aprender-compute/build.rs b/crates/aprender-compute/build.rs index 5c500f8acc..5431c4b79c 100644 --- a/crates/aprender-compute/build.rs +++ b/crates/aprender-compute/build.rs @@ -36,13 +36,10 @@ struct Binding { /// Convert a contract filename + equation into a canonical env var name. fn env_var_name(contract: &str, equation: &str) -> String { - let stem = contract - .trim_end_matches(".yaml") - .trim_end_matches(".yml") - .to_uppercase() - .replace('-', "_"); - let eq = equation.to_uppercase().replace('-', "_"); - format!("CONTRACT_{stem}_{eq}") + provable_contracts::build_helper::env_key( + contract.trim_end_matches(".yaml").trim_end_matches(".yml"), + equation, + ) } /// Rank status values for deduplication. @@ -55,14 +52,8 @@ fn status_rank(s: &str) -> u8 { } fn main() { - let binding_path = Path::new(env!("CARGO_MANIFEST_DIR")) - .join("..") - .join("..") - .join("..") - .join("provable-contracts") - .join("contracts") - .join("trueno") - .join("binding.yaml"); + let binding_path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../contracts/trueno/binding.yaml"); println!("cargo:rerun-if-changed={}", binding_path.display()); diff --git a/crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs b/crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs index 57f7111d77..d4d4cf59ec 100644 --- a/crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs +++ b/crates/aprender-compute/src/backends/gpu/device/linalg/wgsl_forward.rs @@ -108,6 +108,18 @@ pub struct WgslForwardPass { num_kv_heads: u32, head_dim: u32, intermediate_dim: u32, + /// RMSNorm epsilon, passed to the shader as `params.y` (f32 bits). Defaults to + /// [`DEFAULT_RMS_NORM_EPS`]; callers set the model's value with + /// [`WgslForwardPass::set_rms_norm_eps`] (#4056: Llama-family models use 1e-5). + rms_norm_eps: f32, +} + +/// Default RMSNorm epsilon: the constant the shader hardcoded before #4056. +pub const DEFAULT_RMS_NORM_EPS: f32 = 1e-6; + +/// Uniform params for the RMSNorm shader: `(dim, eps as f32 bits, 0, 0)`. +fn rmsnorm_params(dim: u32, eps: f32) -> [u32; 4] { + [dim, eps.to_bits(), 0, 0] } // WGSL shader source for RMSNorm (multi-row via workgroup_id.y) @@ -116,7 +128,7 @@ const RMSNORM_SHADER: &str = r#" @group(0) @binding(0) var input: array; @group(0) @binding(1) var weight: array; @group(0) @binding(2) var output: array; -@group(0) @binding(3) var params: vec4; // (dim, 0, 0, 0) +@group(0) @binding(3) var params: vec4; // (dim, bitcast(eps), 0, 0) var shared_sum: array; @@ -149,7 +161,8 @@ fn main(@builtin(local_invocation_id) lid: vec3, stride >>= 1u; } - let rms = sqrt(shared_sum[0] / f32(dim) + 1e-6); + let eps = bitcast(params.y); + let rms = sqrt(shared_sum[0] / f32(dim) + eps); // Normalize and scale i = tid; @@ -548,9 +561,22 @@ impl WgslForwardPass { num_kv_heads: num_kv_heads as u32, head_dim: head_dim as u32, intermediate_dim: intermediate_dim as u32, + rms_norm_eps: DEFAULT_RMS_NORM_EPS, } } + /// Set the RMSNorm epsilon every norm in this forward pass uses — pass the + /// model's configured `rms_norm_eps`. Default: [`DEFAULT_RMS_NORM_EPS`]. + pub fn set_rms_norm_eps(&mut self, eps: f32) { + self.rms_norm_eps = eps; + } + + /// The RMSNorm epsilon this forward pass uses. + #[must_use] + pub fn rms_norm_eps(&self) -> f32 { + self.rms_norm_eps + } + /// Upload a weight matrix (call once per layer at init). /// PMAT-342: Bias weights (name contains "bias") are stored CPU-side. pub fn upload_weight(&mut self, name: &str, data: &[f32]) { @@ -1954,7 +1980,7 @@ impl WgslForwardPass { output: &wgpu::Buffer, dim: u32, ) { - let params = [dim, 0u32, 0, 0]; + let params = rmsnorm_params(dim, self.rms_norm_eps); let params_buf = self.make_uniform(¶ms); let bg = self.device.create_bind_group(&wgpu::BindGroupDescriptor { label: None, @@ -2156,3 +2182,35 @@ fn bgl_uniform(binding: u32) -> wgpu::BindGroupLayoutEntry { count: None, } } + +#[cfg(test)] +mod rmsnorm_eps_tests { + use super::{rmsnorm_params, DEFAULT_RMS_NORM_EPS}; + + /// #4056: the shader reads eps from `params.y`; it must carry the configured + /// value bit-exactly, not the old hardcoded 1e-6. + #[test] + fn rmsnorm_params_carry_eps_bits() { + let p = rmsnorm_params(896, 1e-5); + assert_eq!(p[0], 896); + assert_eq!(f32::from_bits(p[1]), 1e-5); + assert_ne!(f32::from_bits(p[1]), DEFAULT_RMS_NORM_EPS); + assert_eq!(f32::from_bits(rmsnorm_params(1, DEFAULT_RMS_NORM_EPS)[1]), 1e-6); + } + + /// #4056: the shader now reads `bitcast(params.y)`; naga must accept it. + /// Skips (returns) when no GPU adapter is present. + #[test] + fn rmsnorm_shader_validates_on_device() { + let Ok(gpu) = crate::backends::gpu::GpuDevice::new() else { + return; + }; + gpu.device.push_error_scope(wgpu::ErrorFilter::Validation); + let _module = gpu.device.create_shader_module(wgpu::ShaderModuleDescriptor { + label: Some("rmsnorm_eps_test"), + source: wgpu::ShaderSource::Wgsl(super::RMSNORM_SHADER.into()), + }); + let err = pollster::block_on(gpu.device.pop_error_scope()); + assert!(err.is_none(), "RMSNORM_SHADER failed validation: {err:?}"); + } +} diff --git a/crates/aprender-compute/src/matrix/ops/arithmetic.rs b/crates/aprender-compute/src/matrix/ops/arithmetic.rs index 10b3f77399..46abafec00 100644 --- a/crates/aprender-compute/src/matrix/ops/arithmetic.rs +++ b/crates/aprender-compute/src/matrix/ops/arithmetic.rs @@ -410,7 +410,7 @@ impl Matrix { /// Uses cublasGemmEx with CUBLAS_COMPUTE_32F for numerical safety. #[cfg(feature = "cuda")] fn matmul_cublas(&self, other: &Matrix) -> Result, TruenoError> { - use trueno_gpu::driver::{CublasHandle, CudaContext, CudaStream, GemmOp, GpuBuffer}; + use trueno_gpu::driver::{CublasHandle, CudaContext, CudaStream, GpuBuffer}; let m = self.rows; let k = self.cols; diff --git a/crates/aprender-contracts-cli/Cargo.toml b/crates/aprender-contracts-cli/Cargo.toml index f6104fe6b1..f83d863866 100644 --- a/crates/aprender-contracts-cli/Cargo.toml +++ b/crates/aprender-contracts-cli/Cargo.toml @@ -14,12 +14,18 @@ path = "src/main.rs" [dependencies] aprender-contracts = { path = "../aprender-contracts", version = "0.69.0" } +# #3745 S1: `--model` is declared through the shared role type `ModelPath`, so +# `apr surface` reads `apr pv verify-structure --model` as a model argument. +batuta-common = { workspace = true, features = ["cli-roles"] } clap = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } serde_yaml = { workspace = true } sha2 = { workspace = true } +[build-dependencies] +aprender-build-sha = { workspace = true } # #4219: APR_GIT_SHA for --version + [dev-dependencies] tempfile = "3" diff --git a/crates/aprender-contracts-cli/build.rs b/crates/aprender-contracts-cli/build.rs new file mode 100644 index 0000000000..961a8293c3 --- /dev/null +++ b/crates/aprender-contracts-cli/build.rs @@ -0,0 +1,6 @@ +// #4219: stamp APR_GIT_SHA so `--version` prints ` ()`, +// the shape `apr --version` has carried since #597. Resolution order and +// rerun triggers live in the shared `aprender-build-sha` crate. +fn main() { + build_sha::emit(); +} diff --git a/crates/aprender-contracts-cli/src/cli.rs b/crates/aprender-contracts-cli/src/cli.rs index c790ff16f8..c7d913fc54 100644 --- a/crates/aprender-contracts-cli/src/cli.rs +++ b/crates/aprender-contracts-cli/src/cli.rs @@ -109,6 +109,10 @@ pub enum Commands { /// the tracked contracts.nt / shapes.ttl untouched (aprender#3715) #[arg(long)] out: Option, + /// With `--release-*` and `--surface`: write every DERIVED release cell (the producer's work list, keyed + /// by `cell_id`) to this JSON file (aprender#3745 S2) + #[arg(long)] + cells_out: Option, #[command(flatten)] release: Box, }, @@ -287,9 +291,11 @@ pub enum Commands { /// merge-base(HEAD, origin/main), else the origin/main tip; with neither, NOT CHECKED is printed. #[arg(long)] armed_baseline_ref: Option, - /// Run ONE named gate and report only it (ONT-001 section 5 ONT-2b): `--gate sigma`. + /// Run ONE named gate and report only it (ONT-001 section 5 ONT-2b): `--gate sigma`. Repeatable + /// (PVL-001 EV-11): every named gate runs and reports, and the exit is their meet — a refusal over a + /// reject over a decline over a pass. #[arg(long)] - gate: Option, + gate: Vec, /// With `--gate shapes`: grade only this shape family (the shape and every `.*` shape), armed /// whatever `armed_shapes` says (aprender#3715: `--shape release-readiness-v1`). #[arg(long)] @@ -442,8 +448,9 @@ pub enum Commands { contract_dir: PathBuf, #[arg(long)] config: Option, + /// #3745 S1: typed `ModelPath`, which is how `apr surface` knows this is a model. #[arg(long)] - model: Option, + model: Option, }, /// Verify compositional shape flow across contract dependency graph #[command(name = "verify-pipeline")] @@ -508,6 +515,12 @@ pub struct ReleaseArgs { /// The tokenizer-parity receipts, apr vs the pinned llama.cpp (default: evidence/dogfood/tokenizer//) #[arg(long)] pub tokenizer_receipts: Option, + /// The release candidate's `apr surface --json` (#3745): the release cells are DERIVED from it + #[arg(long)] + pub surface: Option, + /// The CRUX receipts, apr vs its comparators (default: evidence/crux//) + #[arg(long)] + pub crux_receipts: Option, } impl ReleaseArgs { @@ -521,6 +534,8 @@ impl ReleaseArgs { || self.kernel_receipts.is_some() || self.dogfood_receipt.is_some() || self.tokenizer_receipts.is_some() + || self.surface.is_some() + || self.crux_receipts.is_some() } /// The subject, or `None` when no flag was passed. A partial set is refused, never completed by a default. @@ -548,6 +563,8 @@ impl ReleaseArgs { s.dogfood_receipt.clone_from(&self.dogfood_receipt); s.tokenizer_receipts_dir .clone_from(&self.tokenizer_receipts); + s.surface.clone_from(&self.surface); + s.crux_receipts_dir.clone_from(&self.crux_receipts); Ok(Some(s)) } } diff --git a/crates/aprender-contracts-cli/src/commands/extract_rdf.rs b/crates/aprender-contracts-cli/src/commands/extract_rdf.rs index a962e180df..961a1e15bc 100644 --- a/crates/aprender-contracts-cli/src/commands/extract_rdf.rs +++ b/crates/aprender-contracts-cli/src/commands/extract_rdf.rs @@ -33,9 +33,17 @@ pub fn run( check: bool, subject: Option<&Subject>, out: Option<&Path>, + cells_out: Option<&Path>, ) -> Result<(), Box> { if let Some(subject) = subject { - return run_release(contract_dir, check, subject, out); + return run_release(contract_dir, check, subject, out, cells_out); + } + if cells_out.is_some() { + return Err(ReleaseArgsRefused( + "--cells-out lists a release's derived cells; it needs --release-version, --release-commit and --surface" + .into(), + ) + .into()); } if out.is_some() { return Err(ReleaseArgsRefused( @@ -125,6 +133,7 @@ fn run_release( check: bool, subject: &Subject, out: Option<&Path>, + cells_out: Option<&Path>, ) -> Result<(), Box> { if check { return Err(ReleaseArgsRefused( @@ -151,6 +160,9 @@ fn run_release( let mut hasher = Sha256::new(); hasher.update(nt.as_bytes()); std::fs::write(out, &nt)?; + if let Some(path) = cells_out { + write_cells(path, subject, extraction.release.as_ref())?; + } let report = ReleaseExtractReport { triples: extraction.graph.len(), sha256: format!("{:x}", hasher.finalize()), @@ -160,3 +172,26 @@ fn run_release( println!("{}", serde_json::to_string_pretty(&report)?); Ok(()) } + +/// `--cells-out`: every derived cell — the producer's work list, keyed by `cell_id` (aprender#3745 S2). +fn write_cells( + path: &Path, + subject: &Subject, + release: Option<&provable_contracts::ontology::extract::release_evidence::ReleaseStats>, +) -> Result<(), Box> { + let derived = release.map(|r| r.derived.as_slice()).unwrap_or_default(); + if derived.is_empty() { + return Err(ReleaseArgsRefused( + "--cells-out: no cell was derived (no --surface, or a surface with no leaf command)" + .into(), + ) + .into()); + } + let mut doc = serde_json::Map::new(); + doc.insert("schema".into(), "apr-release-cells/v1".into()); + doc.insert("version".into(), subject.version.clone().into()); + doc.insert("release_commit".into(), subject.commit.clone().into()); + doc.insert("cells".into(), serde_json::to_value(derived)?); + std::fs::write(path, serde_json::to_string_pretty(&doc)?)?; + Ok(()) +} diff --git a/crates/aprender-contracts-cli/src/commands/lint.rs b/crates/aprender-contracts-cli/src/commands/lint.rs index 8c9c62febc..b8689acd12 100644 --- a/crates/aprender-contracts-cli/src/commands/lint.rs +++ b/crates/aprender-contracts-cli/src/commands/lint.rs @@ -48,7 +48,7 @@ pub fn run( watch: bool, strict_test_binding: bool, armed_baseline_ref: Option<&str>, - gate: Option<&str>, + gate: &[String], shapes_opts: ShapesOptions, ) -> Result<(), Box> { refuse_missing_corpus(contract_dir)?; @@ -57,8 +57,10 @@ pub fn run( // is asked for, because run_single_gate would otherwise report every ref // missing on exactly the input the refusal exists for. refuse_single_file_strict_binding(contract_dir, strict_test_binding)?; - if let Some(name) = gate { - return run_single_gate(contract_dir, name, &shapes_opts); + match gate { + [] => {} + [name] => return run_single_gate(contract_dir, name, &shapes_opts), + names => return run_gates(contract_dir, names, &shapes_opts), } if watch { return run_watch( @@ -252,6 +254,47 @@ fn run_single_gate( } } +/// PVL-001 EV-11: `--gate a --gate b` runs every named gate, each printing its own report as `--gate a` alone +/// would, and exits with their MEET: any refusal (exit 3) over any reject (1) over any decline (2) over pass (0). +/// A name this build does not compute is refused before any gate runs, so a typo never reports a partial pass. +fn run_gates( + contract_dir: &Path, + names: &[String], + shapes_opts: &ShapesOptions, +) -> Result<(), Box> { + use provable_contracts::lint::NAMED_GATES; + if let Some(bad) = names.iter().find(|n| !NAMED_GATES.contains(&n.as_str())) { + return Err(crate::contract_walk::UnknownGate { + asked: bad.clone(), + known: NAMED_GATES.iter().map(|g| (*g).to_string()).collect(), + } + .into()); + } + let outcomes: Vec<_> = names + .iter() + .map(|n| run_single_gate(contract_dir, n, shapes_opts)) + .collect(); + let rank = |r: &Result<(), Box>| match r { + Ok(()) => 0, + Err(e) if e.is::() => 1, + Err(e) if e.is::() => 2, + Err(_) => 3, + }; + let passed = outcomes.iter().filter(|r| rank(r) == 0).count(); + let worst = outcomes + .into_iter() + .max_by_key(|r| rank(r)) + .unwrap_or(Ok(())); + match worst { + Err(e) if e.is::() => Err(LintRejected { + passed, + armed: names.len(), + } + .into()), + other => other, + } +} + /// One gate's run, mapped to a report or to the refusal/decline that stands in its place. Every non-verdict /// answer says WHY on stderr before it returns, because an exit code without a reason is the thing this gate /// exists to refuse. @@ -266,7 +309,8 @@ fn decide_named_gate( shapes_opts: &ShapesOptions, ) -> Result> { use provable_contracts::lint::{ - relations_gate::RelationsOutcome, sigma_gate::SigmaOutcome, NamedGateOutcome, NAMED_GATES, + ratchet_gates::RatchetOutcome, relations_gate::RelationsOutcome, sigma_gate::SigmaOutcome, + valid_under_gate::ValidUnderOutcome, NamedGateOutcome, NAMED_GATES, }; match provable_contracts::lint::run_named_gate_with(contract_dir, name, shapes_opts) { @@ -292,8 +336,33 @@ fn decide_named_gate( Err(crate::contract_walk::SigmaMalformed(e.to_string()).into()) } NamedGateOutcome::Shapes(outcome) => decide_shapes_gate(outcome), + NamedGateOutcome::ValidUnder(ValidUnderOutcome::NoSigma) => Err(LintDeclined { + reason: provable_contracts::ontology::verdict::Reason::NoCheckable, + } + .into()), + NamedGateOutcome::ValidUnder(ValidUnderOutcome::NoKernels { contracts_checked }) => { + eprintln!( + "valid-under: no kernel-kind contract and no valid_under in {contracts_checked} contract(s) — nothing was measured" + ); + Err(LintDeclined { + reason: provable_contracts::ontology::verdict::Reason::NoCheckable, + } + .into()) + } + NamedGateOutcome::ValidUnder(ValidUnderOutcome::Malformed(e)) => { + Err(crate::contract_walk::SigmaMalformed(e.to_string()).into()) + } + NamedGateOutcome::Ratchet(RatchetOutcome::Declined(why)) => { + eprintln!("{name}: {why}"); + Err(LintDeclined { + reason: provable_contracts::ontology::verdict::Reason::NoCheckable, + } + .into()) + } NamedGateOutcome::Sigma(SigmaOutcome::Ran { result, findings }) + | NamedGateOutcome::Ratchet(RatchetOutcome::Ran { result, findings }) | NamedGateOutcome::Relations(RelationsOutcome::Ran { result, findings }) + | NamedGateOutcome::ValidUnder(ValidUnderOutcome::Ran { result, findings }) | NamedGateOutcome::Ran { result, findings } => Ok((result, findings)), } } @@ -349,6 +418,25 @@ fn decide_shapes_gate( } .into()) } + ShapesOutcome::HarnessBroken { causes } => { + for c in &causes { + eprintln!("shapes: CRUX harness broken — {c}"); + } + Err(LintDeclined { + reason: Reason::NoCheckable, + } + .into()) + } + ShapesOutcome::EmptyDomain { shapes_n } => { + // ONT-4c5 / R-2: |D| = 0 answers nothing about a required cell — a decline, never Pass, never RED + eprintln!( + "shapes: capability-cells domain D is empty ({shapes_n} shape(s)) — no required rung has a host to be measured on" + ); + Err(LintDeclined { + reason: Reason::NoCheckable, + } + .into()) + } ShapesOutcome::PositiveControlFailed { which, .. } => { eprintln!("shapes: positive control {which} did not fire"); Err(LintDeclined { diff --git a/crates/aprender-contracts-cli/src/lib.rs b/crates/aprender-contracts-cli/src/lib.rs index a1050b0411..688eac4738 100644 --- a/crates/aprender-contracts-cli/src/lib.rs +++ b/crates/aprender-contracts-cli/src/lib.rs @@ -22,12 +22,16 @@ use cli::Commands; /// Glance form, printed by `pv -V`. One line, and it names the tool. /// /// clap renders `{name} {version}`, so this yields -/// `pv 0.63.0 (aprender provable-contracts verifier)`. The bare semver stays the -/// SECOND whitespace field because `scripts/pv_bin.sh` reads it positionally to -/// prove a resolved binary was built from HEAD. +/// `pv 0.69.0 (aa7c6ef03) (aprender provable-contracts verifier)` — the build +/// SHA from `aprender-build-sha` (#4219), as every workspace binary prints. +/// The bare semver stays the SECOND whitespace field because +/// `scripts/pv_bin.sh` reads it positionally to prove a resolved binary was +/// built from HEAD. const SHORT_VERSION: &str = concat!( env!("CARGO_PKG_VERSION"), - " (aprender provable-contracts verifier)" + " (", + env!("APR_GIT_SHA"), + ") (aprender provable-contracts verifier)" ); /// Full form, printed by `pv --version`. @@ -41,7 +45,9 @@ const SHORT_VERSION: &str = concat!( /// `tests/version_identity.rs`. const LONG_VERSION: &str = concat!( env!("CARGO_PKG_VERSION"), - " (aprender provable-contracts verifier)\n", + " (", + env!("APR_GIT_SHA"), + ") (aprender provable-contracts verifier)\n", "crate aprender-contracts-cli — ", env!("CARGO_PKG_REPOSITORY"), "\n", @@ -126,12 +132,19 @@ pub fn dispatch(command: Commands) -> Result<(), Box> { contract_dir, check, out, + cells_out, release, } => { let subject = release .subject() .map_err(crate::contract_walk::ReleaseArgsRefused)?; - commands::extract_rdf::run(&contract_dir, check, subject.as_ref(), out.as_deref()) + commands::extract_rdf::run( + &contract_dir, + check, + subject.as_ref(), + out.as_deref(), + cells_out.as_deref(), + ) } Commands::Coverage { contract_dir, @@ -244,8 +257,12 @@ pub fn dispatch(command: Commands) -> Result<(), Box> { watch, strict_test_binding, armed_baseline_ref.as_deref(), - gate.as_deref(), - commands::lint::shapes_options(gate.as_deref(), shape, &release)?, + &gate, + commands::lint::shapes_options( + gate.iter().any(|g| g == "shapes").then_some("shapes"), + shape, + &release, + )?, ) } Commands::Score { diff --git a/crates/aprender-contracts-cli/tests/ev11_lint_ratchets.rs b/crates/aprender-contracts-cli/tests/ev11_lint_ratchets.rs new file mode 100644 index 0000000000..6d76110cf4 --- /dev/null +++ b/crates/aprender-contracts-cli/tests/ev11_lint_ratchets.rs @@ -0,0 +1,219 @@ +//! PVL-001 EV-11 (PMAT-4166) — `pv lint --gate theorem-pairing --gate depends-on-present`: two shrink-only +//! ratchets read from `contracts/lint-baseline.json`, which the gates never write. +//! +//! The row's probe, verbatim (paiml/infra PVL-001 @00553b0b): `json_object contracts/lint-baseline.json && jq -e +//! '(.unpaired_theorem_modules|numbers) and (.contracts_without_depends_on|numbers) and (.command|strings)' +//! contracts/lint-baseline.json && "$PV" lint contracts/ --gate theorem-pairing --gate depends-on-present`, and the +//! accept adds `git diff --exit-code contracts/lint-baseline.json` after it. The first test is that probe on the +//! real corpus; the rest run on a throwaway repo built in a tempdir, because the gates read the repo ROOT +//! (`lean/`, `book/`) as well as the contract dir. +//! +//! | case | expected | +//! |---|---| +//! | real corpus, both gates | exit 0, both Pass, the baseline byte-identical after | +//! | tempdir at the baseline | exit 0 | +//! | the spec's mutation: add an unpaired Theorem module | exit 1, PV-RAT-001 — the meet rejects | +//! | a kernel contract with no `depends_on` above the baseline | exit 1, PV-RAT-002 | +//! | no baseline key | exit 2, `Unknown(Report)` with the count printed — reported, never a pass | +//! | no Lean base | exit 2, decline naming what is missing | +//! | an unknown name among several | exit 1 before anything runs | + +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn pv_bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_pv")) +} + +struct Run { + code: i32, + stdout: String, + stderr: String, +} + +fn pv(args: &[&str]) -> Run { + let scratch = tempfile::tempdir().expect("scratch cwd is creatable"); + let out = Command::new(pv_bin()) + .current_dir(scratch.path()) + .args(args) + .output() + .expect("failed to spawn pv"); + Run { + code: out.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +fn show(r: &Run) -> String { + format!( + "exit {}\n--- stdout\n{}\n--- stderr\n{}", + r.code, r.stdout, r.stderr + ) +} + +fn s(p: &Path) -> &str { + p.to_str().expect("utf-8 path") +} + +fn repo_contracts() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contracts") +} + +fn write(root: &Path, rel: &str, text: &str) { + let p = root.join(rel); + std::fs::create_dir_all(p.parent().expect("has a parent")).expect("mkdir"); + std::fs::write(p, text).expect("write"); +} + +/// `pv lint --gate X` prints one JSON object per gate, pretty-printed and back to back. +fn reports(stdout: &str) -> Vec { + serde_json::Deserializer::from_str(stdout) + .into_iter::() + .map(|v| v.expect("each gate report is JSON")) + .collect() +} + +const BOTH: [&str; 4] = ["--gate", "theorem-pairing", "--gate", "depends-on-present"]; + +/// A repo with one paired theorem module and one kernel contract (the PVL-1 control, which has no +/// `depends_on`), at baselines 0 and 1. +fn repo() -> tempfile::TempDir { + let t = tempfile::tempdir().expect("tempdir"); + write( + t.path(), + "lean/ProvableContracts/Theorems/S/A.lean", + "theorem a : True := trivial\n", + ); + write( + t.path(), + "book/a.md", + "see ProvableContracts.Theorems.S.A\n", + ); + std::fs::create_dir_all(t.path().join("contracts")).expect("mkdir"); + std::fs::copy( + repo_contracts().join("softmax-kernel-v1.yaml"), + t.path().join("contracts/softmax-kernel-v1.yaml"), + ) + .expect("control contract copies"); + write( + t.path(), + "contracts/lint-baseline.json", + "{\n \"unpaired_theorem_modules\": 0,\n \"contracts_without_depends_on\": 1\n}\n", + ); + t +} + +fn lint(root: &Path, gates: &[&str]) -> Run { + let dir = root.join("contracts"); + let mut args = vec!["lint", s(&dir)]; + args.extend_from_slice(gates); + pv(&args) +} + +#[test] +fn the_probe_passes_on_the_real_corpus_and_writes_nothing() { + let baseline = repo_contracts().join("lint-baseline.json"); + let before = std::fs::read(&baseline).expect("baseline readable"); + let doc: serde_json::Value = serde_json::from_slice(&before).expect("baseline is JSON"); + assert!(doc.is_object()); + assert!(doc["unpaired_theorem_modules"].is_u64(), "{doc}"); + assert!(doc["contracts_without_depends_on"].is_u64(), "{doc}"); + assert_eq!(doc["command"], "make lint-ratchet"); + + let dir = repo_contracts(); + let mut args = vec!["lint", s(&dir)]; + args.extend_from_slice(&BOTH); + let r = pv(&args); + assert_eq!(r.code, 0, "{}", show(&r)); + let got = reports(&r.stdout); + assert_eq!(got.len(), 2, "{}", show(&r)); + for g in &got { + assert_eq!(g["verdict"], "Pass", "{g}"); + } + assert_eq!( + got[0]["unpaired_theorem_modules"], doc["unpaired_theorem_modules"], + "the recorded baseline is the measured count (`make lint-ratchet` wrote it)" + ); + assert_eq!( + got[1]["contracts_without_depends_on"], + doc["contracts_without_depends_on"] + ); + assert_eq!( + std::fs::read(&baseline).expect("baseline readable"), + before, + "a gate wrote the baseline" + ); +} + +#[test] +fn at_the_baseline_both_gates_pass() { + let t = repo(); + let r = lint(t.path(), &BOTH); + assert_eq!(r.code, 0, "{}", show(&r)); +} + +/// PVL-001 EV-11's mutation, verbatim: "add an unpaired Theorem module → RED". +#[test] +fn adding_an_unpaired_theorem_module_is_red() { + let t = repo(); + write( + t.path(), + "lean/ProvableContracts/Theorems/S/B.lean", + "theorem b : True := trivial\n", + ); + let r = lint(t.path(), &BOTH); + assert_eq!(r.code, 1, "{}", show(&r)); + assert!(r.stdout.contains("PV-RAT-001"), "{}", show(&r)); + assert!(!r.stdout.contains("PV-RAT-002"), "{}", show(&r)); + assert!( + r.stderr.contains("1/2 armed gates passed"), + "the meet names the one that held: {}", + show(&r) + ); +} + +#[test] +fn a_kernel_contract_without_depends_on_above_the_baseline_is_red() { + let t = repo(); + let text = std::fs::read_to_string(repo_contracts().join("softmax-kernel-v1.yaml")) + .expect("control contract readable"); + write(t.path(), "contracts/softmax-kernel-copy-v1.yaml", &text); + let r = lint(t.path(), &["--gate", "depends-on-present"]); + assert_eq!(r.code, 1, "{}", show(&r)); + assert!(r.stdout.contains("PV-RAT-002"), "{}", show(&r)); +} + +#[test] +fn no_baseline_reports_the_count_and_is_never_a_pass() { + let t = repo(); + write(t.path(), "contracts/lint-baseline.json", "{}\n"); + let r = lint(t.path(), &BOTH); + assert_eq!(r.code, 2, "{}", show(&r)); + let got = reports(&r.stdout); + assert_eq!(got.len(), 2, "{}", show(&r)); + for g in &got { + assert_eq!(g["verdict"], "Unknown(Report)", "{g}"); + } + // the count `make lint-ratchet` records as the first baseline + assert_eq!(got[0]["unpaired_theorem_modules"], 0); + assert_eq!(got[1]["contracts_without_depends_on"], 1); +} + +#[test] +fn no_lean_base_is_a_decline_naming_what_is_missing() { + let t = repo(); + std::fs::remove_dir_all(t.path().join("lean")).expect("rm lean"); + let r = lint(t.path(), &["--gate", "theorem-pairing"]); + assert_eq!(r.code, 2, "{}", show(&r)); + assert!(r.stderr.contains("no Lean theorem base"), "{}", show(&r)); +} + +#[test] +fn an_unknown_name_among_several_is_refused_before_anything_runs() { + let t = repo(); + let r = lint(t.path(), &["--gate", "theorem-pairing", "--gate", "bogus"]); + assert_eq!(r.code, 1, "{}", show(&r)); + assert!(r.stderr.contains("--gate bogus"), "{}", show(&r)); + assert!(r.stdout.is_empty(), "nothing ran: {}", show(&r)); +} diff --git a/crates/aprender-contracts-cli/tests/ont4b_shapes_gate.rs b/crates/aprender-contracts-cli/tests/ont4b_shapes_gate.rs index 1057520206..a55d816909 100644 --- a/crates/aprender-contracts-cli/tests/ont4b_shapes_gate.rs +++ b/crates/aprender-contracts-cli/tests/ont4b_shapes_gate.rs @@ -230,11 +230,13 @@ fn the_tracked_repo_graph_is_fresh() { // that is the ratchet working, not a conflict to route around. It did: the // 0.69 batch folded #3600 in and the count went 6 -> 9 with its three shapes. #3715 added the nine-shape // `release-readiness-v1` family (shapes_n=18, triples=15863, measured on its branch); it contributes no focus - // node to a PR's graph — the release evidence is extracted only under `--release-*`. + // node to a PR's graph — the release evidence is extracted only under `--release-*`. #3745 S2 (#3777) added + // six more for the DERIVED cell classes (effect, probe, model-cell, effect-cell) CRUX (crux-verb, crux) and a8's sampling controls (sampling): 25. + // ONT-4c5 (PMAT-3972) added `capability-cells`: 26. assert_eq!( v["shapes_n"], - 18, - "ont-shapes-v1 + ladder-measured + ladder-green (ONT-4c1) + bound-symbols-resolve + lean-statements-grounded (ONT-4b2) + refusal-receipt-v1 (#3605) + parity-receipt-complete + parity-comparator-self + parity-comparator-oracle (parity-receipt-v2, #3600) + release-readiness-v1{{,.release,.host,.context,.model,.coverage,.tokenizer,.kernel,.refusal}} (#3715)\n{}", + 26, + "ont-shapes-v1 + ladder-measured + ladder-green (ONT-4c1) + bound-symbols-resolve + lean-statements-grounded (ONT-4b2) + refusal-receipt-v1 (#3605) + parity-receipt-complete + parity-comparator-self + parity-comparator-oracle (parity-receipt-v2, #3600) + release-readiness-v1{{,.release,.host,.context,.model,.coverage,.tokenizer,.kernel,.refusal}} (#3715) + release-readiness-v1{{.effect,.probe,.model-cell,.effect-cell,.crux-verb,.crux,.sampling}} (#3745 S2) + capability-cells (ONT-4c5)\n{}", show(&r) ); } diff --git a/crates/aprender-contracts-cli/tests/ont4c3_parity_receipts.rs b/crates/aprender-contracts-cli/tests/ont4c3_parity_receipts.rs index c2452e7955..856a672b9b 100644 --- a/crates/aprender-contracts-cli/tests/ont4c3_parity_receipts.rs +++ b/crates/aprender-contracts-cli/tests/ont4c3_parity_receipts.rs @@ -87,9 +87,18 @@ fn a_complete_receipt_passes_with_one_focus_node() { assert_eq!(r.code, 0, "{}", r.all()); assert_eq!(violations(&r), 0, "{}", r.all()); assert_eq!(focus_nodes(&r), 1, "{}", r.all()); - // The shapes are ARMED in the fixture (no lint-baseline.json narrows them), so a violation here would be - // a real Fail rather than a reported one — which is what makes the two FAIL cases below meaningful. + // The two shapes this receipt reaches are ARMED (the fixture's lint-baseline.json), so a violation here + // would be a real Fail rather than a reported one — which is what makes the two FAIL cases below + // meaningful. `parity-comparator-oracle` is left UNARMED on purpose: a self-comparator receipt gives it + // zero focus nodes, and an ARMED shape that grades nothing declines (#3610) rather than passing. assert!(r.stdout.contains("parity-receipt-complete"), "{}", r.all()); + let v: serde_json::Value = serde_json::from_str(&r.stdout).expect("json report"); + assert_eq!( + names(&v["extra"]["declines"]), + vec!["parity-comparator-oracle"] as Vec<&str>, + "the unarmed vacuity is still NAMED\n{}", + r.all() + ); } #[test] @@ -242,3 +251,10 @@ fn the_committed_tree_agrees_with_its_own_denominator() { String::from_utf8_lossy(&out.stderr) ); } + +/// A JSON array of strings as `Vec<&str>` (`json!` array literals expand to a disallowed `unwrap`). +fn names(v: &serde_json::Value) -> Vec<&str> { + v.as_array() + .map(|a| a.iter().filter_map(serde_json::Value::as_str).collect()) + .unwrap_or_default() +} diff --git a/crates/aprender-contracts-cli/tests/ont4c5_capability_cells.rs b/crates/aprender-contracts-cli/tests/ont4c5_capability_cells.rs new file mode 100644 index 0000000000..6af3402cad --- /dev/null +++ b/crates/aprender-contracts-cli/tests/ont4c5_capability_cells.rs @@ -0,0 +1,371 @@ +//! ONT-4c5 (PMAT-3972; aprender #3972 + #4047) — capability cells, on the CLI. +//! +//! ONT-001 v4.12 §5 ONT-4c5 RED, on `pv lint --gate shapes`: a required cell of the model-capability ladder is +//! Pass or Fail at the current release, and `Unknown{NotRun}` — a missing row, DEFER, MANUAL, NO-VERDICT, a label +//! nobody knows — is a violation of the ARMED `capability-cells` shape: exit 1, naming the cell. An empty domain +//! is a decline (exit 2), never Pass. A Fail cell is ADMITTED by this shape (ladder-green is what refuses it). +//! +//! DISCRIMINATION: `capcells-clean/` must PASS with `pc_shapes` fired and the whole domain reported, so a build +//! that declines or rejects every ladder fails this file; each NotRun fixture must FAIL naming exactly its cell, +//! so a build that arms nothing, or folds an absence into Fail, fails it too; `capcells-failadmit/` must PASS, +//! so a build that treats Fail as NotRun fails it. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +struct Run { + code: i32, + stdout: String, + stderr: String, +} + +fn lint(fixture: &str) -> Run { + let contracts = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../tests/fixtures/ont") + .join(fixture) + .join("contracts"); + let scratch = tempfile::tempdir().expect("scratch cwd is creatable"); + let out = Command::new(PathBuf::from(env!("CARGO_BIN_EXE_pv"))) + .current_dir(scratch.path()) + .args([ + "lint", + &contracts.to_string_lossy(), + "--gate", + "shapes", + "--format", + "json", + ]) + .output() + .expect("failed to spawn pv"); + Run { + code: out.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +fn show(r: &Run) -> String { + format!( + "exit {}\n--- stdout\n{}\n--- stderr\n{}", + r.code, r.stdout, r.stderr + ) +} + +fn json(r: &Run) -> serde_json::Value { + serde_json::from_str(&r.stdout).unwrap_or_else(|e| panic!("not JSON ({e}):\n{}", show(r))) +} + +fn strings(v: &serde_json::Value) -> Vec { + v.as_array() + .unwrap_or_else(|| panic!("not an array: {v}")) + .iter() + .map(|x| x.as_str().expect("string").to_string()) + .collect() +} + +const DOMAIN: [&str; 4] = [ + "rung-a@gx10", + "rung-a@lambda", + "rung-b@gx10", + "rung-b@lambda", +]; + +#[test] +fn a_clean_ladder_passes_with_the_plant_fired_and_the_whole_domain_reported() { + let r = lint("capcells-clean"); + assert_eq!(r.code, 0, "{}", show(&r)); + let v = json(&r); + assert_eq!(v["verdict"], "Pass", "{}", show(&r)); + assert_eq!(v["pc_shapes"]["capability-cells"], "fired", "{}", show(&r)); + assert!(strings(&v["armed_shapes"]).contains(&"capability-cells".to_string())); + assert_eq!(strings(&v["capability_cells"]["domain"]), DOMAIN); + assert!(strings(&v["capability_cells"]["not_run"]).is_empty()); + assert_eq!(v["capability_cells"]["v_star"], "0.69.1"); +} + +/// Every NotRun fixture: exit 1, `not_run` is exactly its cell, and a finding names that cell. +#[test] +fn every_not_run_required_cell_rejects_naming_exactly_that_cell() { + for (fixture, cell) in [ + ("capcells-defer", "rung-a@lambda"), + ("capcells-manual", "rung-a@gx10"), + ("capcells-noverdict", "rung-b@lambda"), + ("capcells-missing", "rung-b@gx10"), + ("capcells-badlabel", "rung-a@lambda"), + ] { + let r = lint(fixture); + assert_eq!(r.code, 1, "{fixture}: {}", show(&r)); + let v = json(&r); + assert_eq!(v["verdict"], "Fail", "{fixture}"); + assert_eq!(v["pc_shapes"]["capability-cells"], "fired", "{fixture}"); + assert_eq!( + strings(&v["capability_cells"]["not_run"]), + [cell], + "{fixture}" + ); + let named = v["findings"].as_array().expect("findings").iter().any(|f| { + f["rule_id"] == "PV-ONT-013" + && f["message"] + .as_str() + .is_some_and(|m| m.contains(&format!("capability cell {cell} "))) + }); + assert!( + named, + "{fixture}: no PV-ONT-013 finding names {cell}\n{}", + show(&r) + ); + } +} + +#[test] +fn an_unknown_label_is_refused_by_name() { + let r = lint("capcells-badlabel"); + let v = json(&r); + let refused = strings(&v["capability_cells"]["refused_labels"]); + assert_eq!(refused.len(), 1, "{}", show(&r)); + assert!(refused[0].ends_with(": rung-a: DEFERRED"), "{refused:?}"); +} + +#[test] +fn a_fail_cell_is_admitted_by_this_shape() { + let r = lint("capcells-failadmit"); + assert_eq!(r.code, 0, "{}", show(&r)); + let v = json(&r); + assert_eq!(v["verdict"], "Pass"); + assert!(strings(&v["capability_cells"]["not_run"]).is_empty()); +} + +#[test] +fn an_empty_domain_declines_never_passes() { + let r = lint("capcells-emptydomain"); + assert_eq!(r.code, 2, "{}", show(&r)); + assert!( + r.stderr.contains("capability-cells domain D is empty"), + "{}", + show(&r) + ); + assert!(r.stderr.contains("decline:"), "{}", show(&r)); +} + +// ── The row's own probe on THIS repository, and the row's data mutations (each must turn it RED) ────────────── +// +// The probe is ONT-4c5's jq predicate, restated: verdict Pass, `capability-cells` armed, `pc_shapes` fired, no +// NotRun cell, and every id of the baseline set B in the domain. The mutations run in a scratch tree that is +// symlinks all the way down except on the path to the one file a mutation edits, which is a real copy — the +// checkout is never written. The row's code mutation ("fold a missing cell into Fail") is held by the unit tests +// and the plant, not here. + +/// ONT-4c5's baseline set B, verbatim from the row's probe (paiml/infra paiml-ontology.md v4.12 §5). +const B: [&str; 14] = [ + "qwen2-1.5b-q4km@lambda", + "qwen2-1.5b-q4km@gx10", + "qwen3-1.7b-q4km@lambda", + "qwen3-1.7b-q4km@gx10", + "qwen35-0.8b-q4km@lambda", + "qwen35-0.8b-q4km@gx10", + "qwen35-2b-q4km@lambda", + "qwen35-2b-q4km@gx10", + "qwen35-4b-q4km@lambda", + "qwen35-4b-q4km@gx10", + "qwen35-9b-q4km@lambda", + "qwen35-9b-q4km@gx10", + "qwen35-27b-q4km@lambda", + "qwen35-27b-q4km@gx10", +]; + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../..") +} + +fn lint_root(root: &Path) -> Run { + let out = Command::new(PathBuf::from(env!("CARGO_BIN_EXE_pv"))) + .current_dir(root) + .args(["lint", "contracts", "--gate", "shapes", "--format", "json"]) + .output() + .expect("failed to spawn pv"); + Run { + code: out.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +/// The row's probe predicate over one run: `Ok(())` green, `Err(why)` red. +fn probe(r: &Run) -> Result<(), String> { + let v: serde_json::Value = + serde_json::from_str(&r.stdout).map_err(|_| format!("no JSON (exit {})", r.code))?; + let has = |key: &str, id: &str| { + v[key] + .as_array() + .is_some_and(|a| a.iter().any(|x| x.as_str() == Some(id))) + }; + if v["verdict"] != "Pass" { + return Err(format!("verdict {}", v["verdict"])); + } + if !has("armed_shapes", "capability-cells") { + return Err("capability-cells not armed".into()); + } + if v["pc_shapes"]["capability-cells"] != "fired" { + return Err("pc_shapes not fired".into()); + } + let cc = &v["capability_cells"]; + if cc["not_run"].as_array().is_none_or(|a| !a.is_empty()) { + return Err(format!("not_run {}", cc["not_run"])); + } + let domain: Vec<&str> = cc["domain"] + .as_array() + .map(|a| a.iter().filter_map(serde_json::Value::as_str).collect()) + .unwrap_or_default(); + let missing: Vec<&str> = B.iter().copied().filter(|b| !domain.contains(b)).collect(); + if missing.is_empty() { + Ok(()) + } else { + Err(format!("B ⊄ D: {missing:?}")) + } +} + +/// The receipt of `host` at the current release V*, as the gate itself reports V* on this repository — never a +/// pinned version, so the mutations follow the tree to whichever release it has receipts for (the branch was +/// parked for a week because these paths named 0.69.1 and its base had only 0.68.2). +fn current_receipt(host: &str) -> String { + let v = json(&lint_root(&repo_root())); + let v_star = v["capability_cells"]["v_star"] + .as_str() + .unwrap_or_else(|| panic!("no v_star on this repository: {v}")); + format!("evidence/dogfood/models/{v_star}/{host}.json") +} + +/// A scratch copy of the repository in which `rel` (and only it) is a real, writable file. +fn scratch_with(rel: &str) -> (tempfile::TempDir, PathBuf) { + let dir = tempfile::tempdir().expect("scratch"); + let repo = repo_root().canonicalize().expect("repo root"); + let rel = Path::new(rel); + let mut src = repo.clone(); + let mut dst = dir.path().to_path_buf(); + let comps: Vec<_> = rel.components().collect(); + for (i, comp) in comps.iter().enumerate() { + // at this level: symlink every sibling, descend into (or copy) the one on the path + for entry in std::fs::read_dir(&src).expect("read_dir") { + let entry = entry.expect("entry"); + let name = entry.file_name(); + if name == comp.as_os_str() || name == ".git" || name == "target" || name == ".pv" { + continue; + } + std::os::unix::fs::symlink(entry.path(), dst.join(&name)).expect("symlink"); + } + src = src.join(comp); + dst = dst.join(comp); + if i + 1 == comps.len() { + std::fs::copy(&src, &dst).expect("copy the mutated file"); + } else { + std::fs::create_dir(&dst).expect("mkdir on the path"); + } + } + (dir, dst) +} + +fn edit_json(path: &Path, f: impl FnOnce(&mut serde_json::Value)) { + let mut v: serde_json::Value = + serde_json::from_str(&std::fs::read_to_string(path).expect("read")).expect("json"); + f(&mut v); + std::fs::write(path, serde_json::to_string_pretty(&v).expect("ser")).expect("write"); +} + +fn rung_row<'a>(v: &'a mut serde_json::Value, id: &str) -> &'a mut serde_json::Value { + v["rungs"] + .as_array_mut() + .expect("rungs") + .iter_mut() + .find(|r| r["id"] == id) + .unwrap_or_else(|| panic!("mutation anchor missing: {id}")) +} + +#[test] +fn the_rows_probe_is_green_on_this_repository() { + let r = lint_root(&repo_root()); + probe(&r).unwrap_or_else(|why| panic!("ONT-4c5 probe RED on the tree: {why}\n{}", show(&r))); +} + +#[test] +fn disarming_capability_cells_turns_the_probe_red() { + let (dir, f) = scratch_with("contracts/lint-baseline.json"); + edit_json(&f, |v| { + let a = v["armed_shapes"].as_array_mut().expect("armed_shapes"); + let n = a.len(); + a.retain(|x| x != "capability-cells"); + assert_eq!(a.len(), n - 1, "mutation anchor missing"); + }); + let r = lint_root(dir.path()); + assert!(probe(&r).is_err(), "disarmed and still green\n{}", show(&r)); + assert_eq!( + r.code, + 2, + "the plant cannot fire through an unarmed shape\n{}", + show(&r) + ); +} + +#[test] +fn flipping_a_required_rung_to_optional_leaves_b_outside_the_domain() { + let (dir, f) = scratch_with("contracts/model-capability-ladder-v1.yaml"); + let text = std::fs::read_to_string(&f).expect("read"); + let at = text + .find("- id: qwen35-2b-q4km") + .expect("mutation anchor missing"); + let tail = &text[at..]; + let next = tail[1..].find("\n - id: ").map_or(tail.len(), |n| n + 1); + let off = tail[..next] + .find("required: true") + .expect("the rung has its own required: true"); + let mut out = text.clone(); + out.replace_range( + at + off..at + off + "required: true".len(), + "required: false", + ); + std::fs::write(&f, out).expect("write"); + let r = lint_root(dir.path()); + let why = probe(&r).expect_err("an optional rung left B ⊆ D intact"); + assert!(why.contains("qwen35-2b-q4km@lambda"), "{why}"); +} + +#[test] +fn a_not_run_label_on_a_current_release_row_rejects_naming_the_cell() { + for label in ["DEFER", "MANUAL", "NO-VERDICT"] { + let (dir, f) = scratch_with(¤t_receipt("lambda")); + edit_json(&f, |v| { + rung_row(v, "qwen35-4b-q4km")["verdict"] = label.into() + }); + let r = lint_root(dir.path()); + assert_eq!(r.code, 1, "{label}\n{}", show(&r)); + assert!(probe(&r).is_err(), "{label}"); + assert!( + r.stdout.contains("qwen35-4b-q4km@lambda"), + "{label}\n{}", + show(&r) + ); + } +} + +#[test] +fn a_deleted_current_release_row_rejects_naming_the_cell() { + let (dir, f) = scratch_with(¤t_receipt("gx10")); + edit_json(&f, |v| { + let rows = v["rungs"].as_array_mut().expect("rungs"); + let n = rows.len(); + rows.retain(|r| r["id"] != "qwen3-1.7b-q4km"); + assert_eq!(rows.len(), n - 1, "mutation anchor missing"); + }); + let r = lint_root(dir.path()); + assert_eq!(r.code, 1, "{}", show(&r)); + assert!(r.stdout.contains("qwen3-1.7b-q4km@gx10"), "{}", show(&r)); +} + +/// Review lane C: the non-numeral-version refusal must reach the PROCESS exit, not only `compute()`'s `Err`. +#[test] +fn a_receipt_version_that_is_not_dotted_numerals_exits_3_naming_the_file() { + let r = lint("capcells-badversion"); + assert_eq!(r.code, 3, "{}", show(&r)); + assert!(r.stderr.contains("0.69.1-rc1"), "{}", show(&r)); + assert!(r.stderr.contains("lambda.json"), "{}", show(&r)); + assert!(r.stderr.contains("refused by name"), "{}", show(&r)); +} diff --git a/crates/aprender-contracts-cli/tests/ont6_lint_verdict.rs b/crates/aprender-contracts-cli/tests/ont6_lint_verdict.rs index 937c1113c8..17c2d1a9b7 100644 --- a/crates/aprender-contracts-cli/tests/ont6_lint_verdict.rs +++ b/crates/aprender-contracts-cli/tests/ont6_lint_verdict.rs @@ -47,9 +47,27 @@ fn repo_contracts() -> PathBuf { Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contracts") } +struct Corpus { + dir: PathBuf, + _root: tempfile::TempDir, +} + +impl Corpus { + fn path(&self) -> &Path { + &self.dir + } +} + /// A corpus directory holding the PVL-1 control contract, and optionally a `lint-baseline.json`. -fn corpus(baseline: Option<&str>) -> tempfile::TempDir { - let d = tempfile::tempdir().expect("corpus dir is creatable"); +/// Nested one level inside a private tempdir: `pv lint` takes the dir's PARENT as the +/// project root, and a bare tempdir's parent is the shared `/tmp` (#4207). +fn corpus(baseline: Option<&str>) -> Corpus { + let root = tempfile::tempdir().expect("corpus dir is creatable"); + let d = Corpus { + dir: root.path().join("contracts"), + _root: root, + }; + std::fs::create_dir(d.path()).expect("nested corpus dir is creatable"); std::fs::copy( repo_contracts().join("softmax-kernel-v1.yaml"), d.path().join("softmax-kernel-v1.yaml"), @@ -109,12 +127,9 @@ fn explicit_empty_armed_set_declines_r2() { #[test] fn a_failing_armed_gate_rejects_at_exit_1() { - let d = tempfile::tempdir().expect("corpus dir"); - std::fs::copy( - repo_contracts().join("softmax-kernel-v1.yaml"), - d.path().join("softmax-kernel-v1.yaml"), - ) - .expect("control copies"); + // Through `corpus()`, not a bare tempdir: on a host with a stray `/tmp/scripts/` + // baseline, PV-DUP-002 would also reject, and this test could pass for the wrong reason. + let d = corpus(None); std::fs::write( d.path().join("broken-v1.yaml"), "metadata:\n version: not-a-contract\n", @@ -263,6 +278,11 @@ fn json_report_carries_the_lattice() { serde_json::Value::String("sigma".into()), serde_json::Value::String("relations".into()), serde_json::Value::String("shapes".into()), + // ONT-7's gate: same R-8 shape. + serde_json::Value::String("valid-under".into()), + // PVL-001 EV-11's two ratchets: same R-8 shape. + serde_json::Value::String("theorem-pairing".into()), + serde_json::Value::String("depends-on-present".into()), ]), "{}", show(&r) diff --git a/crates/aprender-contracts-cli/tests/ont7_valid_under_gate.rs b/crates/aprender-contracts-cli/tests/ont7_valid_under_gate.rs new file mode 100644 index 0000000000..efb38a3a97 --- /dev/null +++ b/crates/aprender-contracts-cli/tests/ont7_valid_under_gate.rs @@ -0,0 +1,234 @@ +//! ONT-7 (PMAT-4076) — `pv lint --gate valid-under`: kernel-kind contracts carry a world index. +//! +//! The row's probe, verbatim (paiml/infra `docs/specifications/paiml-ontology.md` v4.12 :657): +//! `jq -e '.contracts_without_valid_under!=null' contracts/lint-baseline.json && pv lint contracts/ --gate +//! valid-under --format json … jq -e '.verdict=="Pass"'`. The first test is that probe on the real corpus. +//! +//! Every rule has a fixture that fires it, because on the real corpus ONE contract carries `valid_under` +//! (`ont-verdict-lattice-v1`) and a rule that cannot fire there is only observable here: +//! +//! | fixture | expected | +//! |---|---| +//! | `valid-under-ok` | exit 0, Pass — world + every qualifier well-formed | +//! | `valid-under-unknown-world` | exit 1, PV-ONT-014 — `world: mars`, not in Σ | +//! | `valid-under-appendix-b` | exit 0, Pass — the spec's Appendix B example verbatim: no `world`, reads `committed` | +//! | `valid-under-world-not-a-string` | exit 1, PV-ONT-014 | +//! | `valid-under-undeclared-key` | exit 1, PV-ONT-013 — closed key set | +//! | `valid-under-empty` | exit 1, PV-ONT-013 — `valid_under: {}` | +//! | `valid-under-nonkernel-bad` | exit 1, PV-ONT-014 — no kernel at all, and still a reject, not a decline | +//! | `valid-under-bad-qualifier` | exit 1, PV-ONT-015 — `backend: []`, non-string toolchain version | +//! | `valid-under-ratchet-rise` | exit 1, PV-ONT-016 — one kernel without, baseline 0 | +//! | `valid-under-no-kernels` | exit 2, decline — nothing the row obliges | +//! | `sigma-absent` | exit 2, decline — no world index to resolve into | +//! +//! DISCRIMINATION: `valid-under-ok` passes at exit 0 and every reject names its own rule id, so a build that +//! refuses everything, or refuses with the wrong rule, fails this file. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn pv_bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_pv")) +} + +struct Run { + code: i32, + stdout: String, + stderr: String, +} + +fn pv(args: &[&str]) -> Run { + let scratch = tempfile::tempdir().expect("scratch cwd is creatable"); + let out = Command::new(pv_bin()) + .current_dir(scratch.path()) + .args(args) + .output() + .expect("failed to spawn pv"); + Run { + code: out.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +fn show(r: &Run) -> String { + format!( + "exit {}\n--- stdout\n{}\n--- stderr\n{}", + r.code, r.stdout, r.stderr + ) +} + +fn fixture(name: &str) -> String { + let p = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../tests/fixtures/ont") + .join(name); + p.to_str().expect("utf-8 path").to_string() +} + +fn repo_contracts() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contracts") +} + +fn gate(dir: &str) -> Run { + pv(&["lint", dir, "--gate", "valid-under", "--format", "json"]) +} + +fn json_of(r: &Run) -> serde_json::Value { + serde_json::from_str(&r.stdout).unwrap_or_else(|e| panic!("stdout is JSON: {e}\n{}", show(r))) +} + +/// The rejects: exit 1, verdict Fail, and the report names exactly this rule. +fn assert_rejects_with(fixture_name: &str, rule: &str) { + let r = gate(&fixture(fixture_name)); + assert_eq!(r.code, 1, "{fixture_name} must reject\n{}", show(&r)); + assert_eq!(json_of(&r)["verdict"], "Fail", "{}", show(&r)); + assert!( + r.stdout.contains(rule), + "{fixture_name} must reject with {rule}\n{}", + show(&r) + ); + for other in ["PV-ONT-013", "PV-ONT-014", "PV-ONT-015", "PV-ONT-016"] { + if other != rule { + assert!( + !r.stdout.contains(other), + "{fixture_name} fired {other} as well as {rule} — one fixture, one rule\n{}", + show(&r) + ); + } + } +} + +#[test] +fn the_row_probe_passes_on_the_repo_corpus() { + let baseline: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(repo_contracts().join("lint-baseline.json")).expect("baseline"), + ) + .expect("baseline is JSON"); + assert!( + baseline["contracts_without_valid_under"].is_u64(), + "the probe reads a TOP-LEVEL contracts_without_valid_under" + ); + let r = gate(repo_contracts().to_str().expect("utf-8 path")); + assert_eq!(r.code, 0, "{}", show(&r)); + let v = json_of(&r); + assert_eq!(v["gate"], "valid-under", "{}", show(&r)); + assert_eq!(v["verdict"], "Pass", "{}", show(&r)); + let extra = &v["extra"]; + // Not vacuous: the corpus has kernel contracts, and at least one real one resolves its world. + assert!(extra["kernel_contracts"].as_u64() > Some(0), "{}", show(&r)); + assert!( + extra["by_world"] + .as_array() + .expect("by_world") + .iter() + .any(|w| w.as_str().is_some_and(|w| w.starts_with("committed="))), + "ont-verdict-lattice-v1 carries world: committed\n{}", + show(&r) + ); + // The ratchet's own rule, not equality: a PR that annotates contracts lowers the count and passes whether + // or not it also lowers the baseline (as `formal_prose` does), so this never forces a hand-edit. + assert!( + extra["contracts_without_valid_under"].as_u64() <= extra["baseline"].as_u64(), + "the debt may not rise above the recorded baseline\n{}", + show(&r) + ); +} + +#[test] +fn a_well_formed_world_index_passes() { + let r = gate(&fixture("valid-under-ok")); + assert_eq!(r.code, 0, "{}", show(&r)); + let v = json_of(&r); + assert_eq!(v["verdict"], "Pass", "{}", show(&r)); + assert_eq!(v["extra"]["by_world"][0], "committed=1", "{}", show(&r)); +} + +#[test] +fn a_world_sigma_does_not_declare_rejects() { + assert_rejects_with("valid-under-unknown-world", "PV-ONT-014"); +} + +#[test] +fn the_specs_own_appendix_b_example_passes_and_reads_committed() { + let r = gate(&fixture("valid-under-appendix-b")); + assert_eq!(r.code, 0, "{}", show(&r)); + assert_eq!( + json_of(&r)["extra"]["by_world"][0], + "committed=1", + "{}", + show(&r) + ); +} + +#[test] +fn a_world_that_is_not_a_string_rejects() { + assert_rejects_with("valid-under-world-not-a-string", "PV-ONT-014"); +} + +#[test] +fn an_empty_valid_under_rejects() { + assert_rejects_with("valid-under-empty", "PV-ONT-013"); +} + +#[test] +fn a_bad_valid_under_rejects_even_with_no_kernel_contract() { + assert_rejects_with("valid-under-nonkernel-bad", "PV-ONT-014"); +} + +/// R-8: a new ONT gate is COMPUTED in every run and armed per repo, like sigma, relations and shapes. +#[test] +fn the_full_lint_run_computes_valid_under() { + let r = pv(&[ + "lint", + repo_contracts().to_str().expect("utf-8 path"), + "--format", + "json", + ]); + let v = json_of(&r); + let gate = v["gates"] + .as_array() + .expect("gates") + .iter() + .find(|g| g["name"] == "valid-under") + .unwrap_or_else(|| panic!("the full run did not compute valid-under\n{}", show(&r))); + assert_eq!(gate["verdict"], "Pass", "{}", show(&r)); +} + +#[test] +fn an_undeclared_valid_under_key_rejects() { + assert_rejects_with("valid-under-undeclared-key", "PV-ONT-013"); +} + +#[test] +fn a_malformed_qualifier_rejects() { + assert_rejects_with("valid-under-bad-qualifier", "PV-ONT-015"); + let r = gate(&fixture("valid-under-bad-qualifier")); + assert!( + r.stdout.matches("PV-ONT-015").count() >= 2, + "both the empty backend list and the non-string toolchain version are named\n{}", + show(&r) + ); +} + +#[test] +fn a_rise_in_the_debt_rejects() { + assert_rejects_with("valid-under-ratchet-rise", "PV-ONT-016"); +} + +#[test] +fn a_corpus_with_no_kernel_contract_declines() { + let r = gate(&fixture("valid-under-no-kernels")); + assert_eq!( + r.code, + 2, + "zero is a decline, never an accept\n{}", + show(&r) + ); + assert!(r.stderr.contains("no kernel-kind contract"), "{}", show(&r)); +} + +#[test] +fn a_corpus_with_no_sigma_declines() { + let r = gate(&fixture("sigma-absent")); + assert_eq!(r.code, 2, "{}", show(&r)); +} diff --git a/crates/aprender-contracts-cli/tests/ont_release_readiness.rs b/crates/aprender-contracts-cli/tests/ont_release_readiness.rs index 7f60586a5b..94d2755ccb 100644 --- a/crates/aprender-contracts-cli/tests/ont_release_readiness.rs +++ b/crates/aprender-contracts-cli/tests/ont_release_readiness.rs @@ -1,22 +1,25 @@ -//! aprender#3715 — `release-readiness-v1` on the CLI: the red-turning case table. +//! aprender#3715 + #3745 S2 (#3777) — `release-readiness-v1` on the CLI: the red-turning case table over DERIVED +//! cells. //! -//! Every case copies `tests/fixtures/ont/release-green/` AND the real `contracts/release-readiness-v1.yaml` (so the -//! fixture cannot drift from the shapes it proves) into a scratch repo, changes exactly ONE thing, and runs -//! `pv lint contracts --gate shapes --shape release-readiness-v1 --release-version 0.69.1 --release-commit MC …`. +//! Every case copies `tests/fixtures/ont/release-green/` (a measured inventory, a GENERIC surface — `gen` generates, +//! `look` reads a model, `list` takes none; not apr verbs, so S3's hand-list guard has nothing to find) and the +//! real `contracts/release-readiness-v1.yaml` into a scratch repo. The green base then asks pv for the cells it +//! derives (`pv extract … --surface --cells-out`) and writes one passing row per cell, keyed by `cell_id` — so the +//! fixture cannot drift from the derivation — and each case changes exactly ONE thing. //! -//! DISCRIMINATION: the green base must PASS with every cell named (48/48, 4 kernel cells) — so a build that -//! rejects everything fails this file — and each single-cell change must FAIL naming that cell — so a build that -//! accepts everything fails it too. The ONT-001 rule: a shape that has never gone red is not armed. +//! DISCRIMINATION: the green base must PASS with every derived cell carrying a row, so a build that rejects +//! everything fails this file; each single change must FAIL naming what it broke, so a build that accepts +//! everything fails it too. The ONT-001 rule: a shape that has never gone red is not armed. use std::path::{Path, PathBuf}; use std::process::Command; -use serde_json::Value; +use serde_json::{Map, Value}; const MC: &str = "1111111111111111111111111111111111111111"; const PARENT: &str = "3333333333333333333333333333333333333333"; const BUMP: &str = "2222222222222222222222222222222222222222"; -const MODEL: &str = "tiny-qwen35.gguf"; +const GB: u64 = 1_000_000_000; struct Run { code: i32, @@ -27,7 +30,9 @@ struct Run { fn show(r: &Run) -> String { format!( "exit {}\n--- stdout\n{}\n--- stderr\n{}", - r.code, r.stdout, r.stderr + r.code, + r.stdout.chars().take(6000).collect::(), + r.stderr ) } @@ -48,21 +53,6 @@ fn copy_dir(from: &Path, to: &Path) { } } -/// The green base: the fixture plus the REAL shape contract. -fn green() -> tempfile::TempDir { - let t = tempfile::tempdir().expect("tempdir"); - copy_dir( - &repo_root().join("tests/fixtures/ont/release-green"), - t.path(), - ); - std::fs::copy( - repo_root().join("contracts/release-readiness-v1.yaml"), - t.path().join("contracts/release-readiness-v1.yaml"), - ) - .expect("copy the real contract"); - t -} - fn pv(cwd: &Path, args: &[&str]) -> Run { let out = Command::new(env!("CARGO_BIN_EXE_pv")) .current_dir(cwd) @@ -80,35 +70,12 @@ fn s(p: &Path) -> String { p.to_str().expect("utf-8 path").to_string() } -/// The release gate as T-1 calls it, plus `extra` flags. -fn gate(t: &Path, extra: &[&str]) -> Run { - let contracts = s(&t.join("contracts")); - let dogfood = s(&t.join("dogfood-receipt.json")); - let mut args = vec![ - "lint", - &contracts, - "--gate", - "shapes", - "--shape", - "release-readiness-v1", - "--release-version", - "0.69.1", - "--release-commit", - MC, - "--dogfood-receipt", - &dogfood, - ]; - args.extend_from_slice(extra); - pv(t, &args) -} - -fn json_of(r: &Run) -> Value { - serde_json::from_str(&r.stdout).unwrap_or_else(|e| panic!("stdout is JSON: {e}\n{}", show(r))) +fn read(path: &Path) -> Value { + serde_json::from_str(&std::fs::read_to_string(path).expect("read")).expect("json") } fn edit(path: &Path, f: impl FnOnce(&mut Value)) { - let mut v: Value = - serde_json::from_str(&std::fs::read_to_string(path).expect("read")).expect("json"); + let mut v = read(path); f(&mut v); std::fs::write(path, serde_json::to_string_pretty(&v).expect("ser")).expect("write"); } @@ -121,23 +88,218 @@ fn kernels(t: &Path, host: &str) -> PathBuf { t.join(format!("evidence/dogfood/kernels/0.69.1/{host}.json")) } -/// Index of the one row for (verb, thinking, rung) in a model receipt. -fn row_index(v: &Value, verb: &str, thinking: &str, rung: &str) -> usize { - v["cells"] +fn tokenizer(t: &Path) -> PathBuf { + t.join("evidence/dogfood/tokenizer/0.69.1/parity.json") +} + +fn json_of(r: &Run) -> Value { + serde_json::from_str(&r.stdout).unwrap_or_else(|e| panic!("stdout is JSON: {e}\n{}", show(r))) +} + +/// The cells pv derives for the scratch repo, written to `/cells.json` and returned. +fn derive(t: &Path) -> Vec { + let contracts = s(&t.join("contracts")); + let surface = s(&t.join("surface.json")); + let out = s(&t.join("release.nt")); + let cells = s(&t.join("cells.json")); + let r = pv( + t, + &[ + "extract", + &contracts, + "--release-version", + "0.69.1", + "--release-commit", + MC, + "--surface", + &surface, + "--out", + &out, + "--cells-out", + &cells, + ], + ); + assert_eq!(r.code, 0, "{}", show(&r)); + read(&t.join("cells.json"))["cells"] + .as_array() + .expect("cells") + .clone() +} + +/// One PASSING row for a derived cell — whatever its class needs to pass, and nothing it does not. +fn pass_row(c: &Value) -> Value { + let mut r = Map::new(); + r.insert("cell_id".into(), c["id"].clone()); + r.insert("verdict".into(), "pass".into()); + r.insert("answer_chars".into(), 42.into()); + r.insert("rc".into(), 0.into()); + r.insert("wall_ms".into(), 1500.into()); + let generating = c["generates"] == true && c["kind"] == "matrix"; + r.insert( + "backend".into(), + if generating { "cuda" } else { "cpu" }.into(), + ); + r.insert("fallback".into(), false.into()); + if let Some(t) = c["rung_tokens"].as_u64() { + let declared = c["rung"] == "declared"; + r.insert("max_tokens".into(), 256.into()); + let p = if declared { + t.saturating_sub(256) + } else { + t + 1 + }; + r.insert("prompt_tokens".into(), p.into()); + } + if c["thinking"] == "on" { + r.insert("think_closed".into(), true.into()); + } + if generating { + r.insert("f2_source".into(), "fresh".into()); + } + if matches!(c["kind"].as_str(), Some("base" | "effect")) { + let id = c["id"].as_str().unwrap_or_default(); + r.insert("output_sha256".into(), format!("out:{id}").into()); + } + // a8's controls, coherent: T 0 and top-k 1 are the same greedy bytes; seed A twice is identical; seed B differs + if c["kind"] == "sampling" { + let out = match c["control"].as_str() { + Some("t0" | "topk1") => "greedy", + Some("seed-a" | "seed-a-again") => "sampled-a", + _ => "sampled-b", + }; + r.insert("output_sha256".into(), out.into()); + } + Value::Object(r) +} + +/// The green base: the fixture, the REAL contract, and one passing row per derived cell. +fn green() -> tempfile::TempDir { + let t = tempfile::tempdir().expect("tempdir"); + copy_dir( + &repo_root().join("tests/fixtures/ont/release-green"), + t.path(), + ); + std::fs::copy( + repo_root().join("contracts/release-readiness-v1.yaml"), + t.path().join("contracts/release-readiness-v1.yaml"), + ) + .expect("copy the real contract"); + synthesize(t.path()); + t +} + +/// (Re)write every host's rows from the cells pv derives now. +fn synthesize(t: &Path) { + let cells = derive(t); + for h in ["lambda", "gx10"] { + let rows: Vec = cells + .iter() + .filter(|c| c["host"] == h) + .map(pass_row) + .collect(); + edit(&models(t, h), |v| v["cells"] = Value::Array(rows)); + let mut keys: Vec = cells + .iter() + .filter(|c| c["host"] == h && c["command"] == "gen" && c["kind"] == "matrix") + .map(|c| { + format!( + r#"{{"key":{{"model_sha256":{},"host":"{h}","verb":"gen","thinking":{},"rung":{},"prompt_id":"p1"}},"verdict":"GREEN"}}"#, + c["model_sha256"], c["thinking"], c["rung"] + ) + }) + .collect(); + keys.sort(); + keys.dedup(); + // the LAST row is the set's positive control (#3739): one per model per host + if let Some(last) = keys.last_mut() { + *last = last.replace( + r#""verdict":"GREEN""#, + r#""verdict":"GREEN","positive_control":true"#, + ); + } + let crux_dir = t.join("evidence/crux/0.69.1"); + std::fs::create_dir_all(&crux_dir).expect("crux dir"); + let receipt = format!( + r#"{{"schema":"crux-inference-receipt/v1","version":"0.69.1","host":"{h}","apr":{{"version_line":"apr 0.69.1 (111111111)"}},"cells":[{}]}}"#, + keys.join(",") + ); + std::fs::write(crux_dir.join(format!("{h}-gpu.json")), receipt).expect("crux receipt"); + } +} + +/// The id of the first derived cell `pred` accepts. +fn pick(t: &Path, pred: impl Fn(&Value) -> bool) -> String { + read(&t.join("cells.json"))["cells"] .as_array() .expect("cells") .iter() - .position(|c| c["verb"] == verb && c["thinking"] == thinking && c["context"] == rung) - .unwrap_or_else(|| panic!("no row {verb}/{thinking}/{rung}")) + .find(|c| pred(c)) + .and_then(|c| c["id"].as_str()) + .expect("a derived cell matches") + .to_string() +} + +/// The IRI tail a finding names for a cell id: each segment percent-encoded as the graph does. +fn named(id: &str) -> String { + let enc = |seg: &str| { + seg.bytes() + .map(|b| match b { + b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' | b':' => { + (b as char).to_string() + } + _ => format!("%{b:02X}"), + }) + .collect::() + }; + let segs: Vec = id.split('/').map(enc).collect(); + format!("release-cell/0.69.1/{}", segs.join("/")) +} + +/// Edit the one row keyed to `id` on `host`. +fn edit_row(t: &Path, host: &str, id: &str, f: impl FnOnce(&mut Value)) { + edit(&models(t, host), |v| { + let row = v["cells"] + .as_array_mut() + .expect("cells") + .iter_mut() + .find(|r| r["cell_id"] == id) + .expect("the row exists"); + f(row); + }); } -fn row<'a>(v: &'a mut Value, verb: &str, thinking: &str, rung: &str) -> &'a mut Value { - let i = row_index(v, verb, thinking, rung); - &mut v["cells"][i] +fn drop_row(t: &Path, host: &str, id: &str) { + edit(&models(t, host), |v| { + v["cells"] + .as_array_mut() + .expect("cells") + .retain(|r| r["cell_id"] != id); + }); } -fn cell(host: &str, verb: &str, thinking: &str, rung: &str) -> String { - format!("release-cell/0.69.1/{host}/{MODEL}/{verb}/think-{thinking}/{rung}") +/// The release gate as T-1 calls it, plus `extra` flags. +fn gate(t: &Path, extra: &[&str]) -> Run { + let contracts = s(&t.join("contracts")); + let dogfood = s(&t.join("dogfood-receipt.json")); + let surface = s(&t.join("surface.json")); + let mut args = vec![ + "lint", + &contracts, + "--gate", + "shapes", + "--shape", + "release-readiness-v1", + "--release-version", + "0.69.1", + "--release-commit", + MC, + "--dogfood-receipt", + &dogfood, + "--surface", + &surface, + ]; + args.extend_from_slice(extra); + pv(t, &args) } /// RED, and the findings name `needle`. @@ -154,13 +316,29 @@ fn assert_red_naming(r: &Run, needle: &str) -> Value { assert!( msgs.iter().any(|m| m.contains(needle)), "no finding names {needle}:\n{}", - msgs.join("\n") + msgs.iter().take(40).copied().collect::>().join("\n") ); v } +fn gen_cell( + host: &'static str, + thinking: &'static str, + rung: &'static str, +) -> impl Fn(&Value) -> bool { + move |c| { + c["command"] == "gen" + && c["kind"] == "matrix" + && c["host"] == host + && c["thinking"] == thinking + && c["rung"] == rung + } +} + +// ── the green base ─────────────────────────────────────────────────────────────────────────────────────── + #[test] -fn the_green_release_passes_with_every_cell_named() { +fn the_green_release_passes_with_every_derived_cell_carrying_a_row() { let t = green(); let r = gate(t.path(), &[]); assert_eq!(r.code, 0, "{}", show(&r)); @@ -170,96 +348,96 @@ fn the_green_release_passes_with_every_cell_named() { v["pc_shape"], "fired", "the plant fired from the named family" ); + assert_eq!(v["pc_extract"]["release-evidence"], "fired"); + assert_eq!(v["pc_extract"]["cli-surface"], "fired"); + let rel = &v["release"]; + let derived = read(&t.path().join("cells.json"))["cells"] + .as_array() + .map(Vec::len); assert_eq!( - v["pc_extract"]["release-evidence"], "fired", - "the extractor's own control (PMAT-3704)" + rel["cells"].as_u64().map(|n| n as usize), + derived, + "N/N: every derived cell is a node" ); - let rel = &v["release"]; + assert_eq!(rel["cells_with_row"], rel["cells"]); + assert_eq!(rel["orphan_rows"], 0); assert_eq!( - rel["cells"], 48, - "2 hosts × 4 verbs × 2 thinking × (4k, consumer-max, declared)" + rel["probe_cells"], 2, + "`list` takes no model: one probe per host" ); - assert_eq!(rel["cells_with_row"], 48); - assert_eq!(rel["cell_names"].as_array().map(Vec::len), Some(48)); + // `--shape` arms the whole family of sixteen whatever armed_shapes says. The green release has no cell that + // fails to fit, so `.refusal` grades zero focus nodes: it declares `allowEmpty`, so the verdict stands, and + // like every vacuity it is named in `declines` and in neither shape list (#3610). + let armed = v["armed_shapes"].as_array().expect("armed").len(); assert_eq!( - rel["kernel_cells"], 4, - "(q4k_gemv@q4_k, rmsnorm@f32) × 2 hosts" + armed, 15, + "sixteen armed, less the one that measured nothing" ); - assert_eq!(rel["orphan_rows"], 0); assert_eq!( - rel["tokenizer_cells"], 1, - "one distinct model across both hosts (#3726)" + names(&v["declines"]), + vec!["release-readiness-v1.refusal"] as Vec<&str> ); - let armed = v["armed_shapes"].as_array().expect("armed").len(); + assert!( + rel["mode_effects"].as_u64() >= Some(1), + "`gen --chat` is judged by the flag-effect oracle" + ); + assert_eq!(rel["surface"]["generating_commands"], 1); + assert_eq!(rel["kernel_cells"], 4); + assert_eq!(rel["tokenizer_cells"], 1); assert_eq!( - armed, 9, - "--shape arms the whole family whatever armed_shapes says" + rel["sampling_checks"], 2, + "gen's typed sampling args, one model arch, two hosts" ); -} - -#[test] -fn a_missing_cell_is_red_naming_it() { - let t = green(); - edit(&models(t.path(), "lambda"), |v| { - let i = row_index(v, "chat", "on", "consumer-max"); - v["cells"].as_array_mut().expect("cells").remove(i); - }); - assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "chat", "on", "consumer-max"), + assert_eq!(rel["crux"]["verbs"], 3, "every derived leaf verb"); + assert_eq!(rel["crux"]["mapped_verbs"], 1, "only gen has a counterpart"); + assert!(rel["crux"]["obligations"].as_u64() > Some(0)); + let lambda = &rel["projection"]["lambda"]; + assert_eq!( + lambda["unmeasured_cells"], 0, + "every class has a measured median" ); + assert!(lambda["projected_secs"].as_u64() > Some(0), "{lambda}"); } -#[test] -fn a_skipped_cell_is_red_naming_it() { - let t = green(); - edit(&models(t.path(), "gx10"), |v| { - row(v, "serve", "off", "4k")["verdict"] = "skip".into(); - }); - assert_red_naming(&gate(t.path(), &[]), &cell("gx10", "serve", "off", "4k")); -} +// ── #3715's cases, on derived cells ────────────────────────────────────────────────────────────────────── #[test] -fn a_fallback_cell_is_red_naming_it() { +fn a_missing_cell_is_red_naming_it() { let t = green(); - edit(&models(t.path(), "lambda"), |v| { - row(v, "run", "off", "4k")["fallback"] = true.into(); - }); - assert_red_naming(&gate(t.path(), &[]), &cell("lambda", "run", "off", "4k")); + let id = pick(t.path(), gen_cell("lambda", "on", "consumer-max")); + drop_row(t.path(), "lambda", &id); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); } #[test] -fn a_cell_that_does_not_say_whether_it_fell_back_is_red() { - let t = green(); - edit(&models(t.path(), "lambda"), |v| { - row(v, "code", "off", "4k") - .as_object_mut() - .expect("row") - .remove("fallback"); - }); - assert_red_naming(&gate(t.path(), &[]), &cell("lambda", "code", "off", "4k")); +fn a_skipped_a_fallback_or_a_silent_fallback_cell_is_red_naming_it() { + for change in ["skip", "fallback", "silent"] { + let t = green(); + let id = pick(t.path(), gen_cell("gx10", "off", "4k")); + edit_row(t.path(), "gx10", &id, |r| match change { + "skip" => r["verdict"] = "skip".into(), + "fallback" => r["fallback"] = true.into(), + _ => { + r.as_object_mut().expect("row").remove("fallback"); + } + }); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); + } } #[test] -fn a_receipt_measured_at_another_commit_is_stale_and_red() { +fn a_receipt_of_another_commit_or_version_is_stale_and_red() { let t = green(); edit(&models(t.path(), "lambda"), |v| { v["apr_sha"] = PARENT.into() }); let v = assert_red_naming(&gate(t.path(), &[]), "release-host/0.69.1/lambda"); - let msgs = v["findings"].to_string(); - assert!( - msgs.contains(&cell("lambda", "run", "off", "4k")), - "every lambda cell is stale" - ); assert!( - !msgs.contains("release-host/0.69.1/gx10"), + !v["findings"] + .to_string() + .contains("release-host/0.69.1/gx10"), "gx10 is untouched" ); -} - -#[test] -fn a_receipt_for_another_version_is_stale_and_red() { let t = green(); edit(&models(t.path(), "gx10"), |v| { v["version"] = "0.69.0".into() @@ -270,56 +448,25 @@ fn a_receipt_for_another_version_is_stale_and_red() { #[test] fn a_think_on_row_with_an_unclosed_block_and_an_empty_answer_is_red() { let t = green(); - edit(&models(t.path(), "lambda"), |v| { - let r = row(v, "chat", "on", "4k"); + let id = pick(t.path(), gen_cell("lambda", "on", "4k")); + edit_row(t.path(), "lambda", &id, |r| { r["think_closed"] = false.into(); r["answer_chars"] = 0.into(); }); - let v = assert_red_naming(&gate(t.path(), &[]), &cell("lambda", "chat", "on", "4k")); + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); let msgs = v["findings"].to_string(); - assert!(msgs.contains("thinkOk"), "names the unclosed block: {msgs}"); - assert!(msgs.contains("answered"), "names the empty answer: {msgs}"); -} - -#[test] -fn a_missing_think_on_cell_is_red_naming_it() { - let t = green(); - edit(&models(t.path(), "gx10"), |v| { - let i = row_index(v, "code", "on", "declared"); - v["cells"].as_array_mut().expect("cells").remove(i); - }); - assert_red_naming( - &gate(t.path(), &[]), - &cell("gx10", "code", "on", "declared"), + assert!( + msgs.contains("thinkOk") && msgs.contains("answered"), + "{msgs}" ); } #[test] -fn an_absent_declared_length_rung_is_red_and_so_is_an_unmeasured_length() { - let t = green(); - edit(&models(t.path(), "lambda"), |v| { - let i = row_index(v, "run", "off", "declared"); - v["cells"].as_array_mut().expect("cells").remove(i); - }); - assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "run", "off", "declared"), - ); - - // the length is a property of the FILE (one sha256): gx10 dropping it leaves lambda's measurement standing, - // but gx10's own declared cells have no size to fill and go red … +fn a_declared_length_cell_missing_or_an_unmeasured_length_is_red() { let t = green(); - edit(&models(t.path(), "gx10"), |v| { - v["inventory"][0] - .as_object_mut() - .expect("item") - .remove("context_length"); - }); - assert_red_naming( - &gate(t.path(), &[]), - &cell("gx10", "run", "off", "declared"), - ); - // … and with no host measuring it, the model itself is named + let id = pick(t.path(), gen_cell("lambda", "off", "declared")); + drop_row(t.path(), "lambda", &id); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); let t = green(); for h in ["lambda", "gx10"] { edit(&models(t.path(), h), |v| { @@ -333,73 +480,50 @@ fn an_absent_declared_length_rung_is_red_and_so_is_an_unmeasured_length() { } #[test] -fn two_hosts_that_disagree_on_one_files_length_are_red() { +fn a_prompt_short_of_its_rung_or_a_duplicate_row_is_red() { let t = green(); - edit(&models(t.path(), "gx10"), |v| { - v["inventory"][0]["context_length"] = 4096.into(); + let id = pick(t.path(), gen_cell("lambda", "on", "consumer-max")); + edit_row(t.path(), "lambda", &id, |r| { + r["prompt_tokens"] = 1000.into() }); - let v = assert_red_naming(&gate(t.path(), &[]), "contextLength"); - assert!(v["findings"].to_string().contains("maxCount")); -} - -#[test] -fn a_prompt_short_of_its_rung_is_red() { - let t = green(); - edit(&models(t.path(), "lambda"), |v| { - row(v, "serve", "on", "consumer-max")["prompt_tokens"] = 1000.into(); - }); - let v = assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "serve", "on", "consumer-max"), - ); + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); assert!(v["findings"].to_string().contains("contextMet")); -} - -#[test] -fn two_rows_for_one_cell_are_ambiguous_and_red() { let t = green(); + let id = pick(t.path(), gen_cell("lambda", "off", "4k")); edit(&models(t.path(), "lambda"), |v| { - let dup = row(v, "run", "on", "4k").clone(); - v["cells"].as_array_mut().expect("cells").push(dup); + let cells = v["cells"].as_array_mut().expect("cells"); + let dup = cells + .iter() + .find(|r| r["cell_id"] == id.as_str()) + .cloned() + .expect("row"); + cells.push(dup); }); - let v = assert_red_naming(&gate(t.path(), &[]), &cell("lambda", "run", "on", "4k")); + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); assert!(v["findings"].to_string().contains("maxCount")); } #[test] -fn a_kernel_green_on_sm_121_and_red_on_sm_89_is_named_on_lambda_before_any_model_cell() { +fn a_kernel_green_on_sm_121_and_red_on_sm_89_is_named_before_any_cell() { let t = green(); edit(&kernels(t.path(), "lambda"), |v| { v["rows"][0]["verdict"] = "fail".into(); v["rows"][0]["max_err"] = 0.5.into(); }); - // and one model cell too, so the ORDER is observable - edit(&models(t.path(), "lambda"), |v| { - row(v, "run", "off", "4k")["verdict"] = "fail".into(); - }); + let id = pick(t.path(), gen_cell("lambda", "off", "4k")); + edit_row(t.path(), "lambda", &id, |r| r["verdict"] = "fail".into()); let v = assert_red_naming( &gate(t.path(), &[]), "release-kernel/0.69.1/lambda/q4k_gemv/q4_k", ); - let findings = v["findings"].as_array().expect("findings"); - let first = findings[0]["message"].as_str().unwrap_or(""); + let first = v["findings"][0]["message"].as_str().unwrap_or_default(); assert!( first.contains("release-kernel/0.69.1/lambda"), "kernel first: {first}" ); - assert!( - !v["findings"] - .to_string() - .contains("release-kernel/0.69.1/gx10"), - "gx10's kernel is green" - ); -} - -#[test] -fn a_kernel_measured_on_one_arch_only_is_a_missing_row_on_the_other() { let t = green(); edit(&kernels(t.path(), "gx10"), |v| { - v["rows"].as_array_mut().expect("rows").remove(1); // rmsnorm@f32 unmeasured on gx10 + v["rows"].as_array_mut().expect("rows").remove(1); }); assert_red_naming( &gate(t.path(), &[]), @@ -408,22 +532,27 @@ fn a_kernel_measured_on_one_arch_only_is_a_missing_row_on_the_other() { } #[test] -fn a_required_host_with_no_receipt_is_red_naming_the_host() { +fn a_required_host_with_no_receipt_or_a_v1_receipt_is_red_naming_the_host() { let t = green(); std::fs::remove_file(models(t.path(), "gx10")).expect("rm"); assert_red_naming(&gate(t.path(), &[]), "release-host/0.69.1/gx10"); + let t = green(); + edit(&models(t.path(), "gx10"), |v| { + v["schema"] = "apr-model-ladder-receipt/v1".into() + }); + assert_red_naming(&gate(t.path(), &[]), "release-host/0.69.1/gx10"); } #[test] fn an_inherited_dogfood_or_none_at_all_is_red() { let t = green(); edit(&t.path().join("dogfood-receipt.json"), |v| { - v["commit"] = PARENT.into(); + v["commit"] = PARENT.into() }); assert_red_naming(&gate(t.path(), &[]), "release-dogfood/0.69.1"); - let t = green(); let contracts = s(&t.path().join("contracts")); + let surface = s(&t.path().join("surface.json")); let r = pv( t.path(), &[ @@ -437,48 +566,34 @@ fn an_inherited_dogfood_or_none_at_all_is_red() { "0.69.1", "--release-commit", MC, + "--surface", + &surface, ], ); assert_red_naming(&r, "dogfoodReceipt"); } #[test] -fn a_consumer_with_no_recorded_number_is_red_on_the_derived_rung() { +fn a_consumer_with_no_number_or_a_thinking_contradiction_is_red() { let t = green(); edit(&t.path().join("evidence/release/context-rungs.json"), |v| { - v["consumers"].as_array_mut().expect("consumers").push( - serde_json::from_str::( - r#"{"consumer": "pending", "basis": "", "source": "fixture://pending"}"#, - ) - .expect("a consumer record"), - ); + let rec = serde_json::from_str::( + r#"{"consumer":"pending","basis":"","source":"fixture://p"}"#, + ) + .expect("record"); + v["consumers"].as_array_mut().expect("consumers").push(rec); }); assert_red_naming(&gate(t.path(), &[]), "release-context/consumer-max"); -} - -#[test] -fn thinking_that_contradicts_its_template_markers_is_red() { let t = green(); edit(&models(t.path(), "lambda"), |v| { - // off only, though the template carries the `enable_thinking` switch (#3723's derivation says both) v["inventory"][0]["thinking_modes"] = serde_json::from_str::(r#"["off"]"#).expect("modes"); }); assert_red_naming(&gate(t.path(), &[]), "thinkingContradiction"); } -#[test] -fn a_v1_receipt_cannot_stand_for_a_host() { - let t = green(); - edit(&models(t.path(), "gx10"), |v| { - v["schema"] = "apr-model-ladder-receipt/v1".into(); - }); - assert_red_naming(&gate(t.path(), &[]), "release-host/0.69.1/gx10"); -} - #[test] fn t4_receipts_measured_at_the_bump_head_pass_only_with_receipts_commit() { - // T-4: the committed receipts were measured at the bump head; R7 proved it equal to MC modulo evidence/ let t = green(); let other = t.path().join("t4-receipts"); std::fs::create_dir_all(&other).expect("mkdir"); @@ -487,19 +602,171 @@ fn t4_receipts_measured_at_the_bump_head_pass_only_with_receipts_commit() { std::fs::rename(models(t.path(), h), &dst).expect("mv"); edit(&dst, |v| v["apr_sha"] = BUMP.into()); edit(&kernels(t.path(), h), |v| v["apr_sha"] = BUMP.into()); - edit(&tokenizer(t.path()), |v| v["apr_sha"] = BUMP.into()); + } + edit(&tokenizer(t.path()), |v| v["apr_sha"] = BUMP.into()); + for h in ["lambda", "gx10"] { + edit( + &t.path().join(format!("evidence/crux/0.69.1/{h}-gpu.json")), + |v| { + v["apr"]["version_line"] = format!("apr 0.69.1 ({})", &BUMP[..9]).into(); + }, + ); } let dir = s(&other); let r = gate(t.path(), &["--receipts", &dir, "--receipts-commit", BUMP]); assert_eq!(r.code, 0, "{}", show(&r)); - let r = gate(t.path(), &["--receipts", &dir]); - assert_red_naming(&r, "release-host/0.69.1/lambda"); + assert_red_naming( + &gate(t.path(), &["--receipts", &dir]), + "release-host/0.69.1/lambda", + ); } #[test] -fn no_release_subject_declines_it_never_passes() { +fn one_token_id_off_llama_cpp_or_no_comparison_is_red_naming_the_model() { + for change in ["mismatch", "empty", "roundtrip", "absent"] { + let t = green(); + match change { + "mismatch" => edit(&tokenizer(t.path()), |v| { + v["rows"][0]["mismatches"] = 1.into() + }), + "empty" => edit(&tokenizer(t.path()), |v| { + v["rows"][0]["tokens_compared"] = 0.into() + }), + "roundtrip" => edit(&tokenizer(t.path()), |v| { + v["rows"][0]["roundtrip_ok"] = false.into() + }), + _ => std::fs::remove_file(tokenizer(t.path())).expect("rm"), + } + assert_red_naming( + &gate(t.path(), &[]), + "release-tokenizer/0.69.1/tiny-qwen35.gguf", + ); + } +} + +// ── the memory rule (#3710, operator decision "a") ─────────────────────────────────────────────────────── +// 16 GB weights + 1 MB/token KV + 1 GB workspace: `4k` (21.1 GB) fits the 24 GB card; `consumer-max` (25 GB) +// and `declared` (27 GB) do not, and are owed on lambda as honest pre-load refusals — and on gx10 as passes. + +fn memory_split(t: &Path) { + for (h, gpu) in [("lambda", 24 * GB), ("gx10", 119 * GB)] { + edit(&models(t, h), |v| { + v["gpu_mem_total_bytes"] = gpu.into(); + let item = &mut v["inventory"][0]; + item["weights_bytes"] = (16 * GB).into(); + item["kv_bytes_per_token"] = 1_000_000u64.into(); + item["workspace_bytes"] = GB.into(); + }); + } + let cells = read(&t.join("cells.json"))["cells"] + .as_array() + .expect("cells") + .clone(); + edit(&models(t, "lambda"), |v| { + for r in v["cells"].as_array_mut().expect("cells") { + let spec = cells + .iter() + .find(|c| c["id"] == r["cell_id"]) + .expect("spec"); + let need = match (spec["kind"].as_str(), spec["rung"].as_str()) { + (Some("matrix"), Some("consumer-max")) => 25 * GB, + (Some("matrix"), Some("declared")) => 27 * GB, + _ => continue, + }; + r["verdict"] = "refused".into(); + r["required_bytes"] = need.into(); + r["available_bytes"] = (24 * GB).into(); + } + }); +} + +#[test] +fn an_honest_refusal_on_lambda_and_a_pass_on_gx10_is_green() { + let t = green(); + memory_split(t.path()); + let r = gate(t.path(), &[]); + assert_eq!(r.code, 0, "{}", show(&r)); + assert!(json_of(&r)["release"]["refusal_cells"].as_u64() > Some(0)); +} + +#[test] +fn an_oom_a_co_tenant_refusal_or_a_rung_that_fits_nowhere_is_red() { + let t = green(); + memory_split(t.path()); + let id = pick(t.path(), gen_cell("lambda", "on", "declared")); + edit_row(t.path(), "lambda", &id, |r| { + r["verdict"] = "error".into(); + r["reason"] = "CUDA_ERROR_OUT_OF_MEMORY after 41 s of prefill".into(); + }); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); + let t = green(); + memory_split(t.path()); + let id = pick(t.path(), gen_cell("lambda", "off", "consumer-max")); + edit_row(t.path(), "lambda", &id, |r| { + r["required_bytes"] = (23 * GB).into() + }); + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); + assert!(v["findings"].to_string().contains("arithmeticNamed")); + let t = green(); + memory_split(t.path()); + edit(&models(t.path(), "gx10"), |v| { + v["gpu_mem_total_bytes"] = (24 * GB).into() + }); + assert_red_naming( + &gate(t.path(), &[]), + "release-coverage/0.69.1/tiny-qwen35.gguf/declared", + ); +} + +// ── #3745 S2: the cells are DERIVED ─────────────────────────────────────────────────────────────────────── + +#[test] +fn issue_mutant_1_a_new_flag_on_a_generating_command_owes_cells_nobody_has_run() { + let t = green(); + edit(&t.path().join("surface.json"), |v| { + let flag = serde_json::from_str::( + r#"{"id":"extra","long":"extra","value_type":"flag","values":["true","false"],"role":"mode"}"#, + ) + .expect("flag"); + v["commands"][0]["args"] + .as_array_mut() + .expect("args") + .push(flag); + }); + let v = assert_red_naming(&gate(t.path(), &[]), "--extra"); + assert!( + v["findings"] + .to_string() + .contains("release-cell/0.69.1/gen/"), + "the owed cells are gen's" + ); +} + +#[test] +fn issue_mutant_2_the_prompt_with_chat_cell_missing_is_red_naming_it() { + // #3743's shape: `-i file --chat` was covered, the `--prompt` shape was not + let t = green(); + let id = pick(t.path(), |c| { + c["command"] == "gen" + && c["kind"] == "matrix" + && c["host"] == "lambda" + && c["shape"] == "--prompt" + && c["args"].to_string().contains("--chat") + }); + drop_row(t.path(), "lambda", &id); + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); + assert!( + id.contains("--prompt") && id.contains("--chat"), + "the cell names both: {id}" + ); + let _ = v; +} + +#[test] +fn no_surface_or_a_surface_that_does_not_say_what_generates_is_red() { let t = green(); let contracts = s(&t.path().join("contracts")); + let dogfood = s(&t.path().join("dogfood-receipt.json")); let r = pv( t.path(), &[ @@ -509,37 +776,117 @@ fn no_release_subject_declines_it_never_passes() { "shapes", "--shape", "release-readiness-v1", + "--release-version", + "0.69.1", + "--release-commit", + MC, + "--dogfood-receipt", + &dogfood, ], ); + assert_red_naming(&r, "release/surface"); + let t = green(); + edit(&t.path().join("surface.json"), |v| { + v["commands"][0] + .as_object_mut() + .expect("cmd") + .remove("generates"); + }); + assert_red_naming(&gate(t.path(), &[]), "generatesUndeclared"); +} + +#[test] +fn an_unknown_role_arg_beyond_its_ceiling_grows_the_ratchet_and_is_red() { + let t = green(); + edit(&t.path().join("surface.json"), |v| { + let arg = serde_json::from_str::( + r#"{"id":"mystery","long":"mystery","value_type":"text","role":"unknown"}"#, + ) + .expect("arg"); + v["commands"][1]["args"] + .as_array_mut() + .expect("args") + .push(arg); + }); + assert_red_naming(&gate(t.path(), &[]), "ratchetGrew"); +} + +#[test] +fn a_flag_whose_output_never_changes_is_red_naming_the_setting() { + // S2.5: the effect cell prints exactly what its base printed + let t = green(); + let cells = read(&t.path().join("cells.json"))["cells"] + .as_array() + .expect("cells") + .clone(); + for h in ["lambda", "gx10"] { + edit(&models(t.path(), h), |v| { + for r in v["cells"].as_array_mut().expect("cells") { + let spec = cells + .iter() + .find(|c| c["id"] == r["cell_id"]) + .expect("spec"); + if spec["kind"] == "effect" { + r["output_sha256"] = + format!("out:{}", spec["base"].as_str().unwrap_or_default()).into(); + } + } + }); + } + assert_red_naming( + &gate(t.path(), &[]), + "release-effect/0.69.1/gen/--chat%3Dtrue", + ); +} + +#[test] +fn a_modelless_command_must_run_or_refuse_by_name_and_a_model_command_must_pass_or_refuse() { + let t = green(); + let id = pick(t.path(), |c| { + c["command"] == "list" && c["host"] == "lambda" + }); + edit_row(t.path(), "lambda", &id, |r| r["verdict"] = "refused".into()); assert_eq!( - r.code, - 2, - "zero cells is a decline, not a pass: {}", - show(&r) + gate(t.path(), &[]).code, + 0, + "a typed refusal by name is an answer" ); + edit_row(t.path(), "lambda", &id, |r| r["verdict"] = "fail".into()); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); + let t = green(); + let id = pick(t.path(), |c| { + c["command"] == "look" && c["kind"] == "matrix" && c["host"] == "gx10" + }); + drop_row(t.path(), "gx10", &id); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); } +// ── the CLI's own answers ──────────────────────────────────────────────────────────────────────────────── + #[test] -fn a_malformed_subject_or_a_misplaced_flag_is_the_callers_error() { +fn no_release_subject_declines_and_a_malformed_one_is_the_callers_error() { let t = green(); let contracts = s(&t.path().join("contracts")); - let partial = pv( + let r = pv( t.path(), &[ "lint", &contracts, "--gate", "shapes", - "--release-version", - "0.69.1", + "--shape", + "release-readiness-v1", ], ); - assert_eq!(partial.code, 3, "{}", show(&partial)); - let short = pv( - t.path(), - &[ - "lint", - &contracts, + assert_eq!( + r.code, + 2, + "zero cells is a decline, not a pass: {}", + show(&r) + ); + for bad in [ + vec!["--gate", "shapes", "--release-version", "0.69.1"], + vec![ "--gate", "shapes", "--release-version", @@ -547,266 +894,266 @@ fn a_malformed_subject_or_a_misplaced_flag_is_the_callers_error() { "--release-commit", "225b2a9ab", ], - ); - assert_eq!( - short.code, - 3, - "a prefix is refused, never matched: {}", - show(&short) - ); - let misplaced = pv( - t.path(), - &["lint", &contracts, "--gate", "sigma", "--shape", "x"], - ); - assert_eq!(misplaced.code, 3, "{}", show(&misplaced)); - let unknown = pv( - t.path(), - &[ - "lint", - &contracts, - "--gate", - "shapes", - "--shape", - "no-such-shape", - ], - ); - assert_eq!(unknown.code, 3, "{}", show(&unknown)); + vec!["--gate", "sigma", "--shape", "x"], + vec!["--gate", "shapes", "--shape", "no-such-shape"], + ] { + let mut args = vec!["lint", contracts.as_str()]; + args.extend(bad.iter().copied()); + let r = pv(t.path(), &args); + assert_eq!(r.code, 3, "{bad:?}: {}", show(&r)); + } } #[test] -fn pv_extract_writes_the_release_graph_only_where_it_is_told() { +fn pv_extract_writes_the_release_graph_and_the_cell_list_only_where_told() { let t = green(); let contracts = s(&t.path().join("contracts")); - let out = t.path().join("release.nt"); - let dogfood = s(&t.path().join("dogfood-receipt.json")); - let args = [ - "extract", - &contracts, - "--release-version", - "0.69.1", - "--release-commit", - MC, - "--dogfood-receipt", - &dogfood, - ]; - let refused = pv(t.path(), &args); + let refused = pv( + t.path(), + &[ + "extract", + &contracts, + "--release-version", + "0.69.1", + "--release-commit", + MC, + ], + ); assert_eq!(refused.code, 3, "no --out: {}", show(&refused)); + let cells = read(&t.path().join("cells.json")); + assert_eq!(cells["schema"], "apr-release-cells/v1"); + assert_eq!(cells["release_commit"], MC); + let nt = std::fs::read_to_string(t.path().join("release.nt")).expect("written"); + assert!( + nt.contains("release/Cell") && nt.contains("cli/Command"), + "cells and the surface are in the graph" + ); + assert!(!nt.contains("_:"), "no blank nodes (R-15)"); assert!( !t.path().join("contracts/contracts.nt").exists(), "the tracked file is untouched" ); - let out_s = s(&out); - let mut with_out = args.to_vec(); - with_out.extend_from_slice(&["--out", &out_s]); - let r = pv(t.path(), &with_out); - assert_eq!(r.code, 0, "{}", show(&r)); - let v = json_of(&r); - assert_eq!(v["release"]["cells"], 48); - let nt = std::fs::read_to_string(&out).expect("written"); - assert!(nt.contains("release/Cell"), "the cells are in the graph"); - assert!(nt.contains("release/KernelDiffReceipt")); - assert!(!nt.contains("_:"), "no blank nodes (R-15)"); - assert!(!t.path().join("contracts/contracts.nt").exists()); } -fn tokenizer(t: &Path) -> PathBuf { - t.join("evidence/dogfood/tokenizer/0.69.1/parity.json") +// ── S2.4: CRUX for every derived verb (#3739) ─────────────────────────────────────────────────────────── + +fn crux_map(t: &Path) -> PathBuf { + t.join("evidence/crux/verb-correspondence.yaml") } -const TOK_CELL: &str = "release-tokenizer/0.69.1/tiny-qwen35.gguf"; +fn crux_receipt(t: &Path, host: &str) -> PathBuf { + t.join(format!("evidence/crux/0.69.1/{host}-gpu.json")) +} -#[test] -fn one_token_id_that_disagrees_with_llama_cpp_is_red_naming_the_model() { - // #3726: apr mapped every non-ASCII byte to id 0 and `apr parity` could not see it (CPU and GPU share it) - let t = green(); - edit(&tokenizer(t.path()), |v| { - v["rows"][0]["mismatches"] = 1.into() - }); - let v = assert_red_naming(&gate(t.path(), &[]), TOK_CELL); - assert!(v["findings"].to_string().contains("parityExact")); +fn rewrite(path: &Path, from: &str, to: &str) { + let text = std::fs::read_to_string(path).expect("read"); + assert!(text.contains(from), "{from:?} is in {}", path.display()); + std::fs::write(path, text.replace(from, to)).expect("write"); } #[test] -fn a_tokenizer_comparison_over_nothing_or_a_broken_round_trip_or_no_receipt_is_red() { +fn a_verb_without_an_entry_an_engine_it_omits_or_a_none_without_reason_is_red() { let t = green(); - edit(&tokenizer(t.path()), |v| { - v["rows"][0]["tokens_compared"] = 0.into() - }); - assert_red_naming(&gate(t.path(), &[]), TOK_CELL); - + rewrite( + &crux_map(t.path()), + " - verb: look\n comparator: none\n reason: \"fixture: reading a model's header has no comparator counterpart\"\n", + "", + ); + assert_red_naming(&gate(t.path(), &[]), "release-crux-verb/0.69.1/look"); let t = green(); - edit(&tokenizer(t.path()), |v| { - v["rows"][0]["roundtrip_ok"] = false.into() - }); - let v = assert_red_naming(&gate(t.path(), &[]), TOK_CELL); - assert!(v["findings"].to_string().contains("roundTrip")); - + rewrite( + &crux_map(t.path()), + " llamafile: {none: \"fixture: no llamafile build pinned\"}\n", + "", + ); + assert_red_naming(&gate(t.path(), &[]), "engineMissing"); let t = green(); - std::fs::remove_file(tokenizer(t.path())).expect("rm"); - assert_red_naming(&gate(t.path(), &[]), TOK_CELL); + rewrite( + &crux_map(t.path()), + "reason: \"fixture: listing is apr's own\"", + "reason: \"\"", + ); + assert_red_naming(&gate(t.path(), &[]), "noneWithoutReason"); } -// ── the memory rule (#3710, operator decision "a") ────────────────────────────────────────────────────── -// weights 16 GB + 1 MB/token KV + 1 GB workspace: `4k` (21.1 GB) fits the 24 GB card; `consumer-max` (25 GB) and -// `declared` (27 GB) do not, and are owed on lambda as honest pre-load refusals — and on gx10 (119 GB) as passes. - -const GB: u64 = 1_000_000_000; - -fn memory_split(t: &Path) { - for (h, gpu) in [("lambda", 24 * GB), ("gx10", 119 * GB)] { - edit(&models(t, h), |v| { - v["gpu_mem_total_bytes"] = gpu.into(); - let item = &mut v["inventory"][0]; - item["weights_bytes"] = (16 * GB).into(); - item["kv_bytes_per_token"] = 1_000_000u64.into(); - item["workspace_bytes"] = GB.into(); +#[test] +fn a_red_an_unjudged_or_an_absent_crux_cell_is_red_and_all_wrong_is_named_not_failed() { + for verdict in ["RED", "UNJUDGED"] { + let t = green(); + edit(&crux_receipt(t.path(), "lambda"), |v| { + v["cells"][0]["verdict"] = verdict.into() }); + assert_red_naming( + &gate(t.path(), &[]), + "release-crux/0.69.1/lambda/tiny-qwen35.gguf/gen", + ); } - edit(&models(t, "lambda"), |v| { - for c in v["cells"].as_array_mut().expect("cells") { - let need = match c["context"].as_str() { - Some("consumer-max") => 25 * GB, - Some("declared") => 27 * GB, - _ => continue, - }; - c["verdict"] = "refused".into(); - c["required_bytes"] = need.into(); - c["available_bytes"] = (24 * GB).into(); - c["reason"] = - "pre-load refusal: KV for this context needs more than the device holds".into(); - } - }); -} - -#[test] -fn a_27b_that_refuses_honestly_on_lambda_and_passes_on_gx10_is_green() { let t = green(); - memory_split(t.path()); + std::fs::remove_file(crux_receipt(t.path(), "gx10")).expect("rm"); + assert_red_naming( + &gate(t.path(), &[]), + "release-crux/0.69.1/gx10/tiny-qwen35.gguf/gen", + ); + let t = green(); + edit(&crux_receipt(t.path(), "lambda"), |v| { + v["cells"][0]["verdict"] = "ALL_WRONG".into() + }); let r = gate(t.path(), &[]); - assert_eq!(r.code, 0, "{}", show(&r)); - let v = json_of(&r); assert_eq!( - v["release"]["refusal_cells"], 16, - "4 verbs × 2 modes × (consumer-max, declared) on lambda" + r.code, + 0, + "ALL_WRONG is named, not a violation: {}", + show(&r) ); - assert_eq!(v["release"]["cells"], 48, "a refusal is still an owed cell"); + assert_eq!(json_of(&r)["release"]["crux"]["all_wrong"], 1); } +// ── #3748 and the CRUX harness guard ───────────────────────────────────────────────────────────────────── + #[test] -fn the_gx10_cell_that_covers_the_rung_missing_is_red() { +fn a_correctness_cell_validated_by_another_binarys_f2_receipt_is_unmeasured_and_red() { let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "gx10"), |v| { - let i = row_index(v, "chat", "off", "declared"); - v["cells"].as_array_mut().expect("cells").remove(i); + let id = pick(t.path(), gen_cell("lambda", "off", "4k")); + edit_row(t.path(), "lambda", &id, |r| { + r["f2_source"] = "receipt".into(); + r["f2_receipt_binary_sha"] = PARENT.into(); }); - assert_red_naming( - &gate(t.path(), &[]), - &cell("gx10", "chat", "off", "declared"), - ); -} - -#[test] -fn an_oom_mid_run_where_a_refusal_is_owed_is_red() { + let v = assert_red_naming(&gate(t.path(), &[]), &named(&id)); + assert!(v["findings"].to_string().contains("f2Measured")); let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "lambda"), |v| { - let r = row(v, "serve", "on", "declared"); - r["verdict"] = "error".into(); - r["reason"] = "CUDA_ERROR_OUT_OF_MEMORY after 41 s of prefill".into(); - r.as_object_mut().expect("row").remove("required_bytes"); - r.as_object_mut().expect("row").remove("available_bytes"); + let id = pick(t.path(), gen_cell("gx10", "on", "4k")); + edit_row(t.path(), "gx10", &id, |r| { + r["f2_source"] = "receipt".into(); + r["f2_receipt_binary_sha"] = MC.into(); }); - let v = assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "serve", "on", "declared"), + assert_eq!( + gate(t.path(), &[]).code, + 0, + "this binary's own F2 receipt counts" ); - assert!(v["findings"] - .to_string() - .contains("release-readiness-v1.refusal")); + edit_row(t.path(), "gx10", &id, |r| { + r.as_object_mut().expect("row").remove("f2_source"); + }); + assert_red_naming(&gate(t.path(), &[]), &named(&id)); } #[test] -fn a_refusal_whose_arithmetic_shows_no_shortfall_is_red() { +fn a_broken_crux_harness_declines_the_gate_naming_the_model() { let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "lambda"), |v| { - row(v, "code", "off", "consumer-max")["required_bytes"] = (20 * GB).into(); + edit(&crux_receipt(t.path(), "lambda"), |v| { + for c in v["cells"].as_array_mut().expect("cells") { + if c["positive_control"] == true { + c["verdict"] = "ALL_WRONG".into(); + } + } }); - let v = assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "code", "off", "consumer-max"), + let r = gate(t.path(), &[]); + assert_eq!( + r.code, + 2, + "a positive control that is ALL_WRONG declines: {}", + show(&r) + ); + assert!( + r.stderr.contains("tiny-qwen35.gguf") && r.stderr.contains("ALL_WRONG"), + "{}", + show(&r) + ); + let t = green(); + for h in ["lambda", "gx10"] { + edit(&crux_receipt(t.path(), h), |v| { + for c in v["cells"].as_array_mut().expect("cells") { + c["positive_control"] = false.into(); + } + }); + } + let r = gate(t.path(), &[]); + assert_eq!(r.code, 2, "no measured control declines: {}", show(&r)); + assert!( + r.stderr.contains("no measured positive-control cell"), + "{}", + show(&r) ); - assert!(v["findings"].to_string().contains("arithmeticNamed")); } -#[test] -fn a_cell_that_fits_by_the_arithmetic_but_is_missing_is_red() { - let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "lambda"), |v| { - let i = row_index(v, "run", "on", "4k"); - v["cells"].as_array_mut().expect("cells").remove(i); +// ── S2.5: a8's sampling controls over typed `sampling` args ───────────────────────────────────────────── + +/// Set the output digest of `host`'s sampling control `control`. +fn set_control(t: &Path, host: &str, control: &str, out: &str) { + let id = pick(t, |c| { + c["kind"] == "sampling" && c["host"] == host && c["control"] == control }); - assert_red_naming(&gate(t.path(), &[]), &cell("lambda", "run", "on", "4k")); + edit_row(t, host, &id, |r| r["output_sha256"] = out.into()); } #[test] -fn a_rung_that_fits_on_no_required_host_is_red_naming_the_rung() { - let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "gx10"), |v| { - v["gpu_mem_total_bytes"] = (24 * GB).into(); - }); - assert_red_naming( - &gate(t.path(), &[]), - &format!("release-coverage/0.69.1/{MODEL}/declared"), - ); +fn each_of_a8s_sampling_controls_turns_the_gate_red_when_broken() { + let cases: [(&str, &str, &str); 4] = [ + ("seed-a-again", "drifted", "seedRepeatable"), + ("seed-b", "sampled-a", "seedsDiffer"), + ("topk1", "not-greedy", "greedyAgrees"), + ("seed-a", "greedy", "sampledDiffers"), + ]; + for (control, out, check) in cases { + let t = green(); + set_control(t.path(), "lambda", control, out); + if check == "sampledDiffers" { + set_control(t.path(), "lambda", "seed-a-again", "greedy"); + set_control(t.path(), "lambda", "seed-b", "greedy"); + } + let v = assert_red_naming(&gate(t.path(), &[]), "release-sampling/0.69.1/lambda/gen"); + assert!(v["findings"].to_string().contains(check), "{check}"); + } } #[test] -fn a_refusal_because_a_co_tenant_holds_memory_is_red() { - // required 23 GB ≤ the card's 24 GB total, refused because only 20 GB were free: a busy GPU, not the - // arithmetic — it must not shrink what the release proves (#3712, aprender-62) +fn a_control_the_command_cannot_express_is_named_never_passed() { let t = green(); - memory_split(t.path()); - edit(&models(t.path(), "lambda"), |v| { - let r = row(v, "chat", "off", "consumer-max"); - r["required_bytes"] = (23 * GB).into(); - r["available_bytes"] = (20 * GB).into(); + edit(&t.path().join("surface.json"), |v| { + v["commands"][0]["args"] + .as_array_mut() + .expect("args") + .retain(|a| a["sampling_kind"] != "seed"); }); - let v = assert_red_naming( - &gate(t.path(), &[]), - &cell("lambda", "chat", "off", "consumer-max"), + synthesize(t.path()); + let v = assert_red_naming(&gate(t.path(), &[]), "underivable"); + assert!( + v["findings"].to_string().contains("seed-a"), + "names the control it cannot run" ); - assert!(v["findings"].to_string().contains("arithmeticNamed")); } #[test] -fn a_model_whose_template_always_opens_think_owes_only_the_on_cells() { - // Qwen3-style prefill that always opens (#3723): ON only — the OFF cells are not owed, so removing - // every OFF row leaves the release green, and removing an ON row does not +fn a_verb_whose_entry_declares_request_modes_owes_a_crux_cell_in_each() { + // the FILE says which verb has modes (#3739 slice 4); pv never names one let t = green(); + rewrite( + &crux_map(t.path()), + " judged_by: golden\n", + " judged_by: golden\n modes: [nonstream, stream]\n", + ); + let v = assert_red_naming(&gate(t.path(), &[]), "modeMissing"); + assert!(v["findings"].to_string().contains("stream")); for h in ["lambda", "gx10"] { - edit(&models(t.path(), h), |v| { - v["inventory"][0]["thinking_modes"] = - serde_json::from_str::(r#"["on"]"#).expect("modes"); - v["inventory"][0]["thinking_markers"] = - serde_json::from_str::(r#"[""]"#).expect("markers"); - let cells = v["cells"].as_array_mut().expect("cells"); - cells.retain(|c| c["thinking"] == "on"); + edit(&crux_receipt(t.path(), h), |v| { + let rows: Vec = v["cells"].as_array().expect("cells").clone(); + let mut out = Vec::new(); + for r in rows { + for m in ["nonstream", "stream"] { + let mut r2 = r.clone(); + r2["key"]["mode"] = m.into(); + out.push(r2); + } + } + v["cells"] = Value::Array(out); }); } - let r = gate(t.path(), &[]); - assert_eq!(r.code, 0, "{}", show(&r)); - assert_eq!( - json_of(&r)["release"]["cells"], - 24, - "2 hosts × 4 verbs × ON × 3 rungs" - ); - edit(&models(t.path(), "gx10"), |v| { - let i = row_index(v, "code", "on", "4k"); - v["cells"].as_array_mut().expect("cells").remove(i); - }); - assert_red_naming(&gate(t.path(), &[]), &cell("gx10", "code", "on", "4k")); + assert_eq!(gate(t.path(), &[]).code, 0, "a row in each declared mode"); +} + +/// A JSON array of strings as `Vec<&str>` (`json!` array literals expand to a disallowed `unwrap`). +fn names(v: &serde_json::Value) -> Vec<&str> { + v.as_array() + .map(|a| a.iter().filter_map(serde_json::Value::as_str).collect()) + .unwrap_or_default() } diff --git a/crates/aprender-contracts-cli/tests/pvl_zero_contracts.rs b/crates/aprender-contracts-cli/tests/pvl_zero_contracts.rs index ffbe4360c8..3eef41b95b 100644 --- a/crates/aprender-contracts-cli/tests/pvl_zero_contracts.rs +++ b/crates/aprender-contracts-cli/tests/pvl_zero_contracts.rs @@ -105,8 +105,26 @@ fn assert_reported(run: &Run, what: &str, evidence_of_one: &str) { ); } -fn empty_dir() -> tempfile::TempDir { - tempfile::tempdir().expect("empty contract dir is creatable") +/// A contract dir nested one level inside a private tempdir. `pv lint` reads the +/// project root as the dir's PARENT (`scripts/contract_duplicate_stem_baseline.txt`), +/// and a bare tempdir's parent is the shared `/tmp`, so a stray `/tmp/scripts/` +/// failed these tests (#4207). +struct Corpus { + _root: tempfile::TempDir, + dir: PathBuf, +} + +impl Corpus { + fn path(&self) -> &Path { + &self.dir + } +} + +fn empty_dir() -> Corpus { + let root = tempfile::tempdir().expect("empty contract dir is creatable"); + let dir = root.path().join("contracts"); + std::fs::create_dir(&dir).expect("nested contract dir is creatable"); + Corpus { _root: root, dir } } /// A path that does not exist and is never created. diff --git a/crates/aprender-contracts-cli/tests/shapes_vacuity.rs b/crates/aprender-contracts-cli/tests/shapes_vacuity.rs new file mode 100644 index 0000000000..a3264ab7f6 --- /dev/null +++ b/crates/aprender-contracts-cli/tests/shapes_vacuity.rs @@ -0,0 +1,360 @@ +//! #3610 — an armed shape that graded ZERO focus nodes must not be counted as clean. +//! +//! **The defect, measured on pv 0.68.2.** `NoFocus` asked the question GLOBALLY — +//! `report.focus_nodes_n == 0` — which is only ever true for a lone contract. In a **directory** +//! one empty shape is invisible: the other shapes carry the total above zero, the empty one +//! contributes no violations, and the aggregate reports **`Pass` while listing it in +//! `armed_shapes`**. A `type: jsonl` contract whose rows violate its own `sh:pattern` passed exactly +//! that way, and `armed_shapes` is the tool's claim about what it measured. +//! +//! `by_shape` had carried `=0` all along. **Nothing read it** — the same *computed but +//! unexposed* shape as #3597, this time inside the gate itself. +//! +//! **BOTH VENUES, because the lone-contract case already declined correctly and a fix proved only +//! there proves nothing.** The directory case is the one that lied. +//! +//! | fixture | shape reach | armed? | expected | +//! |---|---|---|---| +//! | `shapes-one-empty` | `empty-shape=0`, `tool-status=1` | yes | **exit 2**, decline naming `empty-shape` | +//! | `shapes-one-empty-unarmed` | same | no | **Pass**, `declines: ["empty-shape"]` | +//! | `shapes-one-empty-and-violating` | `empty-shape=0`, `tool-status=1` (violating) | yes | **exit 1**, Fail, `declines: ["empty-shape"]` | +//! | `json-ok` | `tool-status=1` | yes | Pass, `declines: []` | +//! | `shapes-one-empty-allowed` | same, `allowEmpty: ""` on `empty-shape` | yes | **Pass**, `declines: ["empty-shape"]` | +//! | `shapes-all-empty-allowed` | `empty-shape=0` (the only shape), `allowEmpty` | yes | **Pass**, `declines: ["empty-shape"]` | +//! | `shapes-all-empty` | `empty-shape=0` (the only shape) | yes | **exit 2** via the global path, no report yet (#4100) | +//! +//! The second row is what keeps the fix from being "refuse whenever anything is empty": an UNARMED +//! shape at zero did not affect the verdict, so it is **reported** rather than refused — and it is +//! reported, not swallowed, which is the half the original defect got wrong. + +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn pv_bin() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_pv")) +} + +fn repo_root() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../..") +} + +struct Run { + code: i32, + stdout: String, + stderr: String, +} + +impl Run { + fn all(&self) -> String { + format!( + "exit {}\n--- stdout\n{}\n--- stderr\n{}", + self.code, self.stdout, self.stderr + ) + } + fn extra(&self) -> serde_json::Value { + serde_json::from_str::(&self.stdout) + .map(|v| v["extra"].clone()) + .unwrap_or(serde_json::Value::Null) + } +} + +fn shapes_on(fixture: &str) -> Run { + let contracts = repo_root() + .join("tests/fixtures/ont") + .join(fixture) + .join("contracts"); + let out = Command::new(pv_bin()) + .args([ + "lint", + contracts.to_str().expect("utf-8 path"), + "--gate", + "shapes", + "--format", + "json", + ]) + .output() + .expect("failed to spawn pv"); + Run { + code: out.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +#[test] +fn an_armed_shape_that_graded_nothing_refuses_in_a_directory() { + // THE CASE THAT LIED. Before #3610 this was `Pass` with `empty-shape` in `armed_shapes`. + let r = shapes_on("shapes-one-empty"); + assert_eq!( + r.code, + 2, + "a vacuous armed shape must DECLINE, not pass and not fail\n{}", + r.all() + ); + // THE REFUSAL IS THE EXIT CODE; THE REPORT IS THE EVIDENCE. Downstream consumers (infra's SLK + // gate) capture stdout and parse it REGARDLESS of exit code, because pv already exits non-zero + // on Fail. A refusal that printed only a bare `decline:` line would read to them as "no + // by_shape" and score UNMEASURED — indistinguishable, from their side, from a broken pv. + let extra = r.extra(); + assert!( + !extra.is_null(), + "stdout must still parse as JSON on the exit-2 path\n{}", + r.all() + ); + assert_eq!( + serde_json::from_str::(&r.stdout).expect("json")["verdict"].as_str(), + Some("Unknown(NoFocus)"), + "the report must carry the declining verdict\n{}", + r.all() + ); + assert!( + extra["by_shape"] + .as_array() + .expect("by_shape") + .iter() + .any(|s| s.as_str() == Some("empty-shape=0")), + "by_shape must NAME the zero-focus shape on the exit-2 path — it is the evidence\n{}", + r.all() + ); + assert_eq!( + extra["declines"][0].as_str(), + Some("empty-shape"), + "the refusal must name the shape in declines\n{}", + r.all() + ); + assert!( + !extra["armed_shapes"] + .as_array() + .expect("armed_shapes") + .iter() + .any(|s| s.as_str() == Some("empty-shape")), + "a shape that graded nothing has no place in armed_shapes\n{}", + r.all() + ); + assert!( + !extra["not_armed_shapes"] + .as_array() + .expect("not_armed_shapes") + .iter() + .any(|s| s.as_str() == Some("empty-shape")), + "nor in not_armed_shapes: filing a vacuity as a policy choice is how it hid\n{}", + r.all() + ); +} + +#[test] +fn an_unarmed_shape_that_graded_nothing_is_reported_not_refused() { + // The other direction, and the reason this is not "refuse whenever anything is empty". + let r = shapes_on("shapes-one-empty-unarmed"); + assert_eq!( + r.code, + 0, + "an UNARMED empty shape must not refuse\n{}", + r.all() + ); + let extra = r.extra(); + assert_eq!( + extra["declines"].as_array().map(|a| a.len()), + Some(1), + "the empty shape must be REPORTED in declines, not swallowed\n{}", + r.all() + ); + assert_eq!( + extra["declines"][0].as_str(), + Some("empty-shape"), + "{}", + r.all() + ); + assert!( + !extra["armed_shapes"] + .as_array() + .expect("armed_shapes") + .iter() + .any(|s| s.as_str() == Some("empty-shape")), + "a shape that graded nothing has no place in armed_shapes\n{}", + r.all() + ); + // THE HALF THIS TEST WAS MISSING. The criterion is "neither list", and only + // the armed sibling above checked both — so an UNARMED vacuity could sit in + // `not_armed_shapes`, filed as a deliberate policy choice, which is exactly + // how the original defect hid. It did: the partition filtered on + // `vacuous_armed`, and two quorum lanes found it independently while this + // test stayed green. + assert!( + !extra["not_armed_shapes"] + .as_array() + .expect("not_armed_shapes") + .iter() + .any(|s| s.as_str() == Some("empty-shape")), + "an UNARMED shape that graded nothing must not be filed under \ + not_armed_shapes either — `declines` is the only list it belongs in\n{}", + r.all() + ); +} + +/// A REAL VIOLATION OUTRANKS A VACUITY — found by all three lanes of the #3622 re-review. +/// +/// `vacuous_armed` and `passed` are computed over DISJOINT shapes: one armed shape can grade +/// nothing while another armed shape grades a node and finds it broken. The first cut checked +/// vacuity first, so this corpus reported `Unknown(NoFocus)` / exit 2 — and a consumer that reads +/// exit 2 as "unmeasured" never saw a Fail the gate had actually measured. The decline is still +/// REPORTED in `declines`; it just may not hide the verdict. +#[test] +fn a_real_violation_is_a_fail_even_beside_a_vacuous_armed_shape() { + let r = shapes_on("shapes-one-empty-and-violating"); + assert_eq!( + r.code, + 1, + "a measured violation must FAIL, not decline, when another armed shape graded nothing\n{}", + r.all() + ); + assert_eq!( + serde_json::from_str::(&r.stdout).expect("json")["verdict"].as_str(), + Some("Fail"), + "{}", + r.all() + ); + assert_eq!( + r.extra()["declines"][0].as_str(), + Some("empty-shape"), + "the vacuous shape must still be NAMED in declines on the Fail path\n{}", + r.all() + ); +} + +#[test] +fn a_corpus_whose_shapes_all_graded_something_still_passes_with_an_empty_declines() { + // The control for the control: without this, a build that declined EVERYTHING would pass the + // two cases above. + let r = shapes_on("json-ok"); + assert_eq!(r.code, 0, "{}", r.all()); + assert_eq!( + r.extra()["declines"].as_array().map(Vec::len), + Some(0), + "nothing graded zero here, so declines must be empty\n{}", + r.all() + ); +} + +/// THE GLOBAL VENUE — every shape at zero, so `report.focus_nodes_n == 0` and `decline()` returns +/// `ShapesOutcome::NoFocus` before the result path runs. It declined before #3610 and must still. +/// +/// **Corrected by the #3622 re-review (all three lanes).** These two tests first ran +/// `json-missing-ref`, which exits **3** on an unreadable `entity.ref` (`ont_extract_json.rs:128`) +/// before `decline()` is consulted, so `code != 0` and "no report" both held for the wrong reason +/// and neither test ever reached the path it named. `shapes-all-empty` reaches it: exit 2. +#[test] +fn a_corpus_whose_every_shape_graded_nothing_declines_through_the_global_path() { + let r = shapes_on("shapes-all-empty"); + assert_eq!( + r.code, + 2, + "every shape at zero must DECLINE (exit 2), not pass, fail, or error out\n{}", + r.all() + ); + assert!( + r.stderr.contains("NoFocus"), + "the decline must be the NoFocus one, not some other refusal\n{}", + r.all() + ); +} + +/// THE ASYMMETRY, ASSERTED RATHER THAN LEFT UNSAID. The global path renders through +/// `skipped_gate(...)` with no `extra`, so it declines with **no `by_shape` and no `declines`**, +/// while the per-shape path (the one #3610 fixed) declines *with* the full report. The consumer that +/// motivated the report requirement (infra's SLK gate) parses stdout whatever the exit code, so on +/// this path it gets nothing. **Filed as #4100**, not fixed here: it needs the NoFocus outcome to +/// carry the report, a change to the outcome path rather than to the verdict. +#[test] +fn the_global_nofocus_decline_prints_no_report_and_that_is_recorded() { + let r = shapes_on("shapes-all-empty"); + assert_eq!(r.code, 2, "{}", r.all()); + assert!( + r.extra().is_null(), + "if this venue has GAINED a report, #4100 is fixed — replace this test with an assertion of \ + the report and add `shapes-all-empty` to the_three_answers_remain_distinct's loop\n{}", + r.all() + ); +} + +#[test] +fn the_three_answers_remain_distinct() { + // pass / fail / decline must not collapse: a build that returned one code for everything + // would satisfy any single assertion above. + let pass = shapes_on("json-ok").code; + let fail = shapes_on("json-violation").code; + let decline = shapes_on("shapes-one-empty").code; + assert_eq!( + (pass, fail, decline), + (0, 1, 2), + "pass / fail / decline must differ" + ); + // Every one of the three must also PRINT its report — a consumer that captures stdout and + // parses it regardless of exit code must get a document in all three cases, not two. + for fixture in ["json-ok", "json-violation", "shapes-one-empty"] { + let r = shapes_on(fixture); + assert!( + !r.extra().is_null(), + "{fixture} did not emit a parseable report\n{}", + r.all() + ); + assert!( + r.extra()["by_shape"].is_array(), + "{fixture}: extra.by_shape must be present whatever the exit code\n{}", + r.all() + ); + } +} + +#[test] +fn an_armed_shape_declared_allow_empty_is_named_but_does_not_refuse() { + // A target class that is empty BY DESIGN (a release:RefusalCell exists only when a cell does not fit) + // declares it, with its reason, in the artifact. The exemption is for the verdict only: the vacuity is + // still named, so a reader sees the gate looked at nothing for it. + let r = shapes_on("shapes-one-empty-allowed"); + assert_eq!( + r.code, + 0, + "an allowEmpty shape must not refuse\n{}", + r.all() + ); + let extra = r.extra(); + assert_eq!( + names(&extra["declines"]), + vec!["empty-shape"] as Vec<&str>, + "the vacuity is still NAMED\n{}", + r.all() + ); + assert_eq!( + names(&extra["armed_shapes"]), + vec!["tool-status"] as Vec<&str>, + "{}", + r.all() + ); + // The control: the SAME fixture without the declaration declines. + assert_eq!(shapes_on("shapes-one-empty").code, 2); +} + +/// A JSON array of strings as `Vec<&str>` (`json!` array literals expand to a disallowed `unwrap`). +fn names(v: &serde_json::Value) -> Vec<&str> { + v.as_array() + .map(|a| a.iter().filter_map(serde_json::Value::as_str).collect()) + .unwrap_or_default() +} + +#[test] +fn a_lone_allow_empty_shape_passes_through_the_global_path_too() { + // Quorum lane C (#3610): the GLOBAL NoFocus early-return (every shape in scope at zero) fired before the + // per-shape exemption was consulted, so a contract whose ONLY shape is empty by design still exited 2 — + // with no report at all. Every shape declaring allowEmpty now reaches the per-shape path. + let r = shapes_on("shapes-all-empty-allowed"); + assert_eq!(r.code, 0, "{}", r.all()); + assert_eq!( + names(&r.extra()["declines"]), + vec!["empty-shape"], + "{}", + r.all() + ); + // The control: the same lone shape WITHOUT the declaration still declines through the global path. + assert_eq!(shapes_on("shapes-all-empty").code, 2); +} diff --git a/crates/aprender-contracts-macros/src/lib.rs b/crates/aprender-contracts-macros/src/lib.rs index 5a7da9d198..40610c3cee 100644 --- a/crates/aprender-contracts-macros/src/lib.rs +++ b/crates/aprender-contracts-macros/src/lib.rs @@ -23,7 +23,14 @@ //! `CONTRACT__=bound` env vars for each implemented binding. //! //! 2. `#[contract("name", equation = "eq")]` expands to a `const` that reads -//! the corresponding env var via `env!()`. Missing env var = compile error. +//! the corresponding env var via `option_env!()`. A missing env var is a +//! compile error ONLY when the producer says its registry ran +//! (`CONTRACT_BINDING_SOURCE=binding.yaml`, #4368). Without that sentinel +//! (a crates.io build, or a producer that found no binding.yaml) the +//! attribute enforces nothing (#2699). Assertions +//! are injected only for the `_PRE_*` / `_POST_*` vars a producer emitted. +//! Producers derive the key with `provable_contracts::build_helper::env_key`, +//! which is tested equal to [`contract_env_key!`], the key this macro reads. //! //! 3. A static string in a dedicated link section registers the binding for //! runtime audit (when `contract-audit` feature is enabled). @@ -96,8 +103,12 @@ impl Parse for ContractArgs { /// Annotates a function with a provable-contracts YAML contract reference. /// The macro generates: /// -/// 1. A `const` assertion that reads a `CONTRACT__` env var -/// (set by build.rs). If the env var is missing, compilation fails. +/// 1. A `const` that reads a `CONTRACT__` env var (set by +/// build.rs) via `option_env!`. If the env var is missing and the +/// producer set `CONTRACT_BINDING_SOURCE=binding.yaml`, that is a +/// `compile_error!` naming the site (#4368). With no such sentinel, +/// NOTHING fails: the binding is unverified and no assertion is injected +/// (#2699). /// /// 2. `debug_assert!()` calls for EVERY precondition and postcondition /// from the YAML contract (read via `CONTRACT__PRE_N` env vars). @@ -119,7 +130,9 @@ impl Parse for ContractArgs { /// /// This macro reads those env vars at compile time and injects the /// assertions. Change the YAML → assertions change automatically. -/// Remove the YAML → compile error. +/// Remove the YAML → the assertions silently disappear (#2699). A condition +/// the producer DID emit but that does not parse as a Rust expression, or a +/// `_COUNT` whose numbered vars are missing, is a compile error. /// /// # Example /// @@ -154,6 +167,9 @@ pub fn contract(attr: TokenStream, item: TokenStream) -> TokenStream { args.equation_name.to_uppercase().replace(['-', '.'], "_") ); + // A registry that ran and has no entry for this site (#4368). + let registry_miss = registry_miss(&env_key, contract_name, equation_name); + // Read preconditions from env vars set by build.rs let precondition_asserts = read_contract_assertions(&env_key, "PRE", equation_name); @@ -217,6 +233,7 @@ pub fn contract(attr: TokenStream, item: TokenStream) -> TokenStream { }; let expanded = quote! { + #registry_miss #(#fn_attrs)* #fn_vis #fn_sig { #body @@ -226,42 +243,113 @@ pub fn contract(attr: TokenStream, item: TokenStream) -> TokenStream { TokenStream::from(expanded) } +/// The env var a producer build.rs sets to `binding.yaml` once it has read +/// its binding registry and emitted a `CONTRACT__` var for every row. +const REGISTRY_SENTINEL: &str = "CONTRACT_BINDING_SOURCE"; + +/// `compile_error!` when the producer's registry ran (the sentinel reads +/// `binding.yaml`) but has nothing for this site: no binding var, no pre +/// count, no post count. That is a misspelt or unregistered contract, which +/// used to expand to nothing, exactly like a crates.io build with no +/// producer. With no sentinel (or `none`) this stays silent (#4368). +fn registry_miss( + env_key: &str, + contract: &str, + equation: &str, +) -> Option { + if std::env::var(REGISTRY_SENTINEL).ok().as_deref() != Some("binding.yaml") { + return None; + } + let known = ["", "_PRE_COUNT", "_POST_COUNT"] + .iter() + .any(|suffix| std::env::var(format!("{env_key}{suffix}")).is_ok()); + (!known).then(|| { + compile_error(&format!( + "#[contract({contract}, equation = \"{equation}\")]: this crate's binding registry has no \ + row for it ({env_key} is unset while {REGISTRY_SENTINEL}=binding.yaml) -- add the binding or fix the name (#4368)" + )) + }) +} + /// Read CONTRACT__{PRE,POST}_0..N env vars and generate `debug_assert`! tokens. /// /// build.rs sets these from YAML contract preconditions/postconditions. -/// If no env vars are found (e.g., crates.io build), returns empty vec (graceful). +/// No `_COUNT` var (e.g. a crates.io build, where no build.rs ran) yields no +/// assertions. Once a producer HAS emitted a count, every inconsistency in +/// what it emitted is a `compile_error!` rather than a silently dropped +/// assertion (#2699): an unparseable count, a missing numbered var, or a +/// condition that is not a Rust expression. fn read_contract_assertions( env_key: &str, kind: &str, // "PRE" or "POST" equation_name: &str, ) -> Vec { - let mut asserts = Vec::new(); - let count_key = format!("{env_key}_{kind}_COUNT"); - let count: usize = std::env::var(&count_key) - .ok() - .and_then(|v| v.parse().ok()) - .unwrap_or(0); + let Ok(raw_count) = std::env::var(&count_key) else { + return Vec::new(); + }; + let Ok(count) = raw_count.parse::() else { + return vec![compile_error(&format!( + "{count_key}={raw_count:?} is not a count (#2699)" + ))]; + }; + (0..count) + .map(|i| condition_assert(&format!("{env_key}_{kind}_{i}"), kind, equation_name)) + .collect() +} +/// One `debug_assert!` for the condition in env var `var_key`, or a +/// `compile_error!` naming why the producer's condition cannot be enforced. +fn condition_assert(var_key: &str, kind: &str, equation_name: &str) -> proc_macro2::TokenStream { + let Ok(expr_str) = std::env::var(var_key) else { + return compile_error(&format!( + "{var_key} is missing although its _COUNT covers it (#2699)" + )); + }; + let Ok(expr) = syn::parse_str::(&expr_str) else { + return compile_error(&format!( + "{var_key}={expr_str:?} is not a Rust expression; the contract condition \ + would be dropped (#2699)" + )); + }; let kind_label = if kind == "PRE" { "Pre" } else { "Post" }; - - for i in 0..count { - let var_key = format!("{env_key}_{kind}_{i}"); - if let Ok(expr_str) = std::env::var(&var_key) { - // Parse the YAML precondition/postcondition as a Rust expression - if let Ok(expr) = expr_str.parse::() { - let msg = format!( - "Contract [{equation_name}] {kind_label}-condition violated: {expr_str}" - ); - asserts.push(quote! { - debug_assert!(#expr, #msg); - }); - } - // If parsing fails, skip silently (the expression may not be valid Rust) - } + let msg = format!("Contract [{equation_name}] {kind_label}-condition violated: {expr_str}"); + quote! { + debug_assert!(#expr, #msg); } +} - asserts +fn compile_error(msg: &str) -> proc_macro2::TokenStream { + quote! { ::core::compile_error!(#msg); } +} + +/// The env var key `#[contract(contract, equation = equation)]` reads, as a +/// string literal: `contract_env_key!("rmsnorm-kernel-v1", "rmsnorm")` is +/// `"CONTRACT_RMSNORM_KERNEL_V1_RMSNORM"`. The conditions live under that key +/// plus `_PRE_COUNT`, `_PRE_`, `_POST_COUNT` and `_POST_`. +/// +/// It exists so a producer's key derivation can be TESTED against the one +/// this crate reads, instead of re-derived by hand and trusted (#2699 §4). +#[proc_macro] +pub fn contract_env_key(input: TokenStream) -> TokenStream { + let args = parse_macro_input!(input as KeyArgs); + let key = make_env_key(&args.contract.value(), &args.equation.value()); + TokenStream::from(quote! { #key }) +} + +/// Arguments to `contract_env_key!("contract-name", "equation-name")`. +struct KeyArgs { + contract: syn::LitStr, + equation: syn::LitStr, +} + +impl Parse for KeyArgs { + fn parse(input: ParseStream<'_>) -> syn::Result { + let contract = input.parse()?; + input.parse::()?; + let equation = input.parse()?; + Ok(KeyArgs { contract, equation }) + } } /// Precondition: checked via `debug_assert!()` at function entry. @@ -429,4 +517,89 @@ mod tests { fn test_make_env_key_with_dots() { assert_eq!(make_env_key("v1.0", "eq.1"), "CONTRACT_V1_0_EQ_1"); } + + /// (env vars under the case's key, expected (asserts, compile_errors)). + /// A producer that emitted nothing is the crates.io case and stays + /// silent; everything a producer DID emit is enforced or refused (#2699). + #[test] + fn read_contract_assertions_fails_closed_on_what_a_producer_emitted() { + let cases: [(&str, &[(&str, &str)], (usize, usize)); 6] = [ + ("NONE", &[], (0, 0)), + ( + "OK", + &[ + ("PRE_COUNT", "2"), + ("PRE_0", "x > 0"), + ("PRE_1", "!v.is_empty()"), + ], + (2, 0), + ), + ("BADCOUNT", &[("PRE_COUNT", "two")], (0, 1)), + ("GAP", &[("PRE_COUNT", "2"), ("PRE_0", "x > 0")], (1, 1)), + ( + "UNPARSEABLE", + &[("PRE_COUNT", "1"), ("PRE_0", "x >")], + (0, 1), + ), + ( + "NOT_AN_EXPR", + &[("PRE_COUNT", "1"), ("PRE_0", "x > 0, \"msg\"")], + (0, 1), + ), + ]; + for (name, vars, (want_asserts, want_errors)) in cases { + let key = format!("CONTRACT_TEST_2699_{name}"); + for (suffix, value) in vars { + std::env::set_var(format!("{key}_{suffix}"), value); + } + let out: Vec = read_contract_assertions(&key, "PRE", "eq") + .iter() + .map(ToString::to_string) + .collect(); + let errors = out.iter().filter(|t| t.contains("compile_error")).count(); + let asserts = out.iter().filter(|t| t.contains("debug_assert")).count(); + assert_eq!( + (asserts, errors), + (want_asserts, want_errors), + "{name}: {out:?}" + ); + } + } + /// #4368: with the sentinel at `binding.yaml`, a site the registry has no + /// row for is a compile error; any one of the three vars makes it known. + /// Without the sentinel (a crates.io build), or at `none`, it is silent. + /// The only test that touches the sentinel, so it owns it for its run. + #[test] + fn registry_miss_is_an_error_only_when_the_registry_ran() { + let cases: [(&str, Option<&str>, &[&str], bool); 7] = [ + ("NO_SENTINEL", None, &[], false), + ("SENTINEL_NONE", Some("none"), &[], false), + ("MISS", Some("binding.yaml"), &[], true), + ("BOUND", Some("binding.yaml"), &[""], false), + ("PRE_ONLY", Some("binding.yaml"), &["_PRE_COUNT"], false), + ("POST_ONLY", Some("binding.yaml"), &["_POST_COUNT"], false), + ("OTHER_KEY", Some("binding.yaml"), &[], true), + ]; + for (name, sentinel, suffixes, want_error) in cases { + match sentinel { + Some(v) => std::env::set_var(REGISTRY_SENTINEL, v), + None => std::env::remove_var(REGISTRY_SENTINEL), + } + let key = format!("CONTRACT_TEST_4368_{name}"); + for suffix in suffixes { + std::env::set_var(format!("{key}{suffix}"), "1"); + } + if name == "OTHER_KEY" { + // A neighbouring row with a shared prefix does not make this one known. + std::env::set_var(format!("{key}X"), "implemented"); + } + let out = registry_miss(&key, "c", "e").map(|t| t.to_string()); + assert_eq!( + out.as_deref().is_some_and(|t| t.contains("compile_error")), + want_error, + "{name}: {out:?}" + ); + } + std::env::remove_var(REGISTRY_SENTINEL); + } } diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts.lean b/crates/aprender-contracts-staging/lean/ProvableContracts.lean index b6035b1cde..2b974836f0 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts.lean @@ -108,3 +108,66 @@ import ProvableContracts.Theorems.Softmax.PartitionOfUnity import ProvableContracts.Theorems.Softmax.ShiftInvariance import ProvableContracts.Theorems.Sparse.SpMVLinear import ProvableContracts.Theorems.Transpose.Involution + +-- Modules nothing imported, so `lake build` never compiled them and leanchecker never +-- re-checked their theorems (#4244). scripts/check_lean_modules_reachable.sh keeps every +-- module under ProvableContracts/ reachable from here. +import ProvableContracts.Defs.GPU +import ProvableContracts.Defs.Image +import ProvableContracts.Defs.Rand +import ProvableContracts.Defs.Tensor +import ProvableContracts.Theorems.AdamW.Adam_Moments +import ProvableContracts.Theorems.AdamW.Adam_Variance +import ProvableContracts.Theorems.AdamW.Analytic +import ProvableContracts.Theorems.AdamW.Bias_Correction +import ProvableContracts.Theorems.AdamW.Weight_Update +import ProvableContracts.Theorems.AprCode.HarnessIrRoundtrip +import ProvableContracts.Theorems.BLAS.Trmm +import ProvableContracts.Theorems.BLAS.Trsm +import ProvableContracts.Theorems.CrossEntropy.CoreSoftmaxInvariants +import ProvableContracts.Theorems.DPO +import ProvableContracts.Theorems.F16.Conversion +import ProvableContracts.Theorems.FFT.Bluestein +import ProvableContracts.Theorems.FFT.Fft2d +import ProvableContracts.Theorems.FFT.Fft3d +import ProvableContracts.Theorems.FFT.FftBatched +import ProvableContracts.Theorems.Fusion.Correctness +import ProvableContracts.Theorems.GPU.DimensionIndependence +import ProvableContracts.Theorems.GatedDeltaNet.Recurrence +import ProvableContracts.Theorems.GgufExportSymmetry.Roundtrip +import ProvableContracts.Theorems.Image.Canny +import ProvableContracts.Theorems.Image.ConnectedComponents +import ProvableContracts.Theorems.Image.Conv2d +import ProvableContracts.Theorems.Image.Histogram +import ProvableContracts.Theorems.Image.HsvRoundtrip +import ProvableContracts.Theorems.Image.Morphology +import ProvableContracts.Theorems.Image.Resize +import ProvableContracts.Theorems.Image.RgbToGray +import ProvableContracts.Theorems.Image.Sobel +import ProvableContracts.Theorems.LoRA.Dare_Unbiased +import ProvableContracts.Theorems.LoRA.Lora_Shape +import ProvableContracts.Theorems.LoRA.Shape_Preservation +import ProvableContracts.Theorems.LoRA.Task_Vector +import ProvableContracts.Theorems.Lora.MergeForwardEquiv +import ProvableContracts.Theorems.MatMul.CooperativeTiling +import ProvableContracts.Theorems.MatMul.Distributivity +import ProvableContracts.Theorems.MatMul.MatVecLinearity +import ProvableContracts.Theorems.MatMul.Shape +import ProvableContracts.Theorems.MatMul.TransposeMul +import ProvableContracts.Theorems.Metrics.Regression +import ProvableContracts.Theorems.Metrics.RegressionAnalytic +import ProvableContracts.Theorems.Quantization.NF4Dequant +import ProvableContracts.Theorems.RMSNorm.NormalizedRMS +import ProvableContracts.Theorems.Rand.Philox +import ProvableContracts.Theorems.Rand.Threefry +import ProvableContracts.Theorems.Rope.Rope +import ProvableContracts.Theorems.Sigmoid.SiluAsymptotic +import ProvableContracts.Theorems.Sigmoid.SiluSign +import ProvableContracts.Theorems.Sparse.BsrSpmv +import ProvableContracts.Theorems.Sparse.SellSpmv +import ProvableContracts.Theorems.Sparse.Spgemm +import ProvableContracts.Theorems.Sparse.Spmm +import ProvableContracts.Theorems.Tensor.Einsum +import ProvableContracts.Theorems.TensorLayout.IndexAlgebra +import ProvableContracts.Theorems.TensorTranspose.Roundtrip +import ProvableContracts.Theorems.Tokenizer diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/GPU.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/GPU.lean index 0096a4ca95..c0fb8b540b 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/GPU.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/GPU.lean @@ -11,7 +11,7 @@ namespace ProvableContracts.GPU /-- A dimension-independent kernel produces the same PTX regardless of input dimensions. We model this as: the kernel function is parametric in dimensions (they are runtime params, not compile-time). -/ -def dimension_independent (kernel : ℕ → ℕ → ℕ → α) : Prop := +def dimension_independent {α : Type*} (kernel : ℕ → ℕ → ℕ → α) : Prop := ∀ m₁ k₁ n₁ m₂ k₂ n₂, kernel m₁ k₁ n₁ = kernel m₂ k₂ n₂ end ProvableContracts.GPU diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/Rand.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/Rand.lean index 83c97f822b..11a8280a15 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/Rand.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Defs/Rand.lean @@ -15,11 +15,11 @@ namespace ProvableContracts.Rand /-- A counter-based RNG is a pure function from (key, counter) to output. Determinism is the core property: same inputs → same output. -/ -def deterministic (f : α → β → γ) : Prop := +def deterministic {α β γ : Type*} (f : α → β → γ) : Prop := ∀ k c, f k c = f k c /-- Counter increment preserves key. -/ -def counter_independent (f : α → ℕ → γ) : Prop := - ∀ k c₁ c₂, c₁ ≠ c₂ → True -- We can't prove outputs differ without the actual function +def counter_independent {α γ : Type*} (f : α → ℕ → γ) : Prop := + ∀ (k : α) (c₁ c₂ : ℕ), c₁ ≠ c₂ → True -- We can't prove outputs differ without the actual function end ProvableContracts.Rand diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean index e2dd4c1532..c3d629ee7f 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Alibi/Slopes.lean @@ -56,7 +56,7 @@ theorem alibi_head_zero_eight : alibiSlope 8 0 = 1 / 2 := by -- Status: proved /-- Every ALiBi slope is strictly positive (base `2 > 0`). -/ -theorem alibi_slope_pos (n h : ℝ) : 0 < alibiSlope n h := by +theorem alibi_slope_pos_real (n h : ℝ) : 0 < alibiSlope n h := by unfold alibiSlope exact Real.rpow_pos_of_pos (by norm_num) _ @@ -97,13 +97,13 @@ theorem alibi_slope_strict_anti {n h₁ h₂ : ℝ} (hn : 0 < n) (h : h₁ < h -- Tests #check @alibi_head_zero #check @alibi_head_zero_eight -#check @alibi_slope_pos +#check @alibi_slope_pos_real #check @alibi_slope_lt_one #check @alibi_slope_ggml #check @alibi_slope_strict_anti example : alibiSlope 8 0 = 1 / 2 := alibi_head_zero_eight -example : (0 : ℝ) < alibiSlope 8 3 := alibi_slope_pos 8 3 +example : (0 : ℝ) < alibiSlope 8 3 := alibi_slope_pos_real 8 3 example : alibiSlope 8 0 = alibiM0 8 := alibi_head_zero 8 end ProvableContracts.Alibi diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/AprCode/HarnessIrRoundtrip.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/AprCode/HarnessIrRoundtrip.lean index 4c80fea001..17d44c008a 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/AprCode/HarnessIrRoundtrip.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/AprCode/HarnessIrRoundtrip.lean @@ -95,7 +95,7 @@ theorem list_roundtrip (bs : List Block) (h : ∀ b ∈ bs, anthNative b) : induction bs with | nil => rfl | cons b bs ih => - have hb : anthNative b := h b (List.mem_cons_self b bs) + have hb : anthNative b := h b (List.mem_cons_self) have hbs : ∀ b' ∈ bs, anthNative b' := fun b' hb' => h b' (List.mem_cons_of_mem b hb') simp only [List.map_cons, block_roundtrip b hb, ih hbs] @@ -151,7 +151,7 @@ theorem gem_list_roundtrip (bs : List Block) (h : ∀ b ∈ bs, gemNative b) : induction bs with | nil => rfl | cons b bs ih => - have hb : gemNative b := h b (List.mem_cons_self b bs) + have hb : gemNative b := h b (List.mem_cons_self) have hbs : ∀ b' ∈ bs, gemNative b' := fun b' hb' => h b' (List.mem_cons_of_mem b hb') simp only [List.map_cons, gem_block_roundtrip b hb, ih hbs] @@ -188,7 +188,7 @@ theorem cross_harness_list (bs : List Block) (h : ∀ b ∈ bs, sharedCore b) : induction bs with | nil => rfl | cons b bs ih => - have hb : sharedCore b := h b (List.mem_cons_self b bs) + have hb : sharedCore b := h b (List.mem_cons_self) have hbs : ∀ b' ∈ bs, sharedCore b' := fun b' hb' => h b' (List.mem_cons_of_mem b hb') simp only [List.map_cons, cross_harness_block b hb, ih hbs] diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trmm.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trmm.lean index f64a33b9b3..e3146789df 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trmm.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trmm.lean @@ -1,3 +1,4 @@ +import Mathlib.Data.Real.Basic import Mathlib.Data.Matrix.Basic /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trsm.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trsm.lean index f9ae2067d0..bd91081e4b 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trsm.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/BLAS/Trsm.lean @@ -1,3 +1,4 @@ +import Mathlib.Data.Real.Basic import Mathlib.Data.Matrix.Basic import Mathlib.LinearAlgebra.Matrix.NonsingularInverse diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft2d.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft2d.lean index 6b91b6d487..bf867b1749 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft2d.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft2d.lean @@ -1,3 +1,4 @@ +import Mathlib.Analysis.Complex.Exponential import Mathlib.Data.Complex.Basic import Mathlib.Algebra.BigOperators.Group.Finset.Basic diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft3d.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft3d.lean index 7c3369345b..8a50e1165e 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft3d.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/Fft3d.lean @@ -1,3 +1,4 @@ +import Mathlib.Analysis.Complex.Exponential import Mathlib.Data.Complex.Basic /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/FftBatched.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/FftBatched.lean index c92aa54650..38eab5c207 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/FftBatched.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/FFT/FftBatched.lean @@ -12,7 +12,7 @@ namespace ProvableContracts.FFT -- Status: proved /-- Batch independence: function applied to i-th element doesn't depend on j-th. -/ -theorem batch_independence {n : ℕ} (f : Fin n → α) (i : Fin n) : +theorem batch_independence {n : ℕ} {α : Type*} (f : Fin n → α) (i : Fin n) : f i = f i := rfl #check @batch_independence diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/GPU/DimensionIndependence.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/GPU/DimensionIndependence.lean index 43604168da..1b865bd18d 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/GPU/DimensionIndependence.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/GPU/DimensionIndependence.lean @@ -12,7 +12,7 @@ namespace ProvableContracts.GPU -- Status: proved /-- A constant kernel is trivially dimension-independent. -/ -theorem const_is_dimension_independent (c : α) : +theorem const_is_dimension_independent {α : Type*} (c : α) : dimension_independent (fun (_ _ _ : ℕ) => c) := by intro m₁ k₁ n₁ m₂ k₂ n₂ rfl diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Canny.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Canny.lean index 9771577e17..08b2d5e9de 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Canny.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Canny.lean @@ -1,3 +1,4 @@ +import Mathlib.Tactic.Linarith import Mathlib.Data.Real.Basic /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Histogram.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Histogram.lean index 593a65f8c3..1bcce58404 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Histogram.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Histogram.lean @@ -1,3 +1,4 @@ +import Mathlib.Data.Fintype.Card import ProvableContracts.Defs.Image import Mathlib.Algebra.BigOperators.Group.Finset.Basic diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Morphology.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Morphology.lean index 60f9483dcd..d18906ff3f 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Morphology.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Morphology.lean @@ -1,4 +1,4 @@ -import Mathlib.Order.BooleanAlgebra +import Mathlib.Order.BooleanAlgebra.Basic /-! # Morphology — Opening is Anti-Extensive diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Resize.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Resize.lean index 7893fd3774..930302718c 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Resize.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/Resize.lean @@ -1,3 +1,4 @@ +import Mathlib.Tactic.Linarith import Mathlib.Data.Real.Basic /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/RgbToGray.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/RgbToGray.lean index ee2b233c8c..163b284958 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/RgbToGray.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Image/RgbToGray.lean @@ -1,3 +1,4 @@ +import Mathlib.Tactic.NormNum import ProvableContracts.Defs.Image /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Quantization/NF4Dequant.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Quantization/NF4Dequant.lean index f0f527e9ea..c252e4b5d8 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Quantization/NF4Dequant.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Quantization/NF4Dequant.lean @@ -28,11 +28,11 @@ namespace ProvableContracts.NF4 axiom nf4_lut : Fin 16 → ℝ /-- NF4 dequantization of a single nibble. -/ -def dequant_nibble (nibble : Fin 16) : ℝ := +noncomputable def dequant_nibble (nibble : Fin 16) : ℝ := nf4_lut nibble /-- Blockwise dequantization: x_i = LUT[nibble_i] * absmax[i / blocksize] -/ -def dequant_blockwise (nibbles : List (Fin 16)) (absmax : List ℝ) (blocksize : ℕ) +noncomputable def dequant_blockwise (nibbles : List (Fin 16)) (absmax : List ℝ) (blocksize : ℕ) (_hbs : blocksize > 0) : List ℝ := (nibbles.zip (List.range nibbles.length)).map fun ⟨n, i⟩ => dequant_nibble n * (absmax.getD (i / blocksize) 0) diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Philox.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Philox.lean index 8868ed1a0c..c7ac9b30ae 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Philox.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Philox.lean @@ -15,7 +15,7 @@ namespace ProvableContracts.Rand -- Status: proved /-- Any pure function is deterministic. -/ -theorem philox_deterministic (f : α → β → γ) : deterministic f := by +theorem philox_deterministic {α β γ : Type*} (f : α → β → γ) : deterministic f := by intro k c rfl diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Threefry.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Threefry.lean index e64f602ab0..81c84e4d54 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Threefry.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Rand/Threefry.lean @@ -15,7 +15,7 @@ namespace ProvableContracts.Rand -- Status: proved /-- ThreeFry is deterministic (same inputs → same output). -/ -theorem threefry_deterministic (f : α → β → γ) : deterministic f := by +theorem threefry_deterministic {α β γ : Type*} (f : α → β → γ) : deterministic f := by intro k c; rfl #check @threefry_deterministic diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sigmoid/SiluAsymptotic.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sigmoid/SiluAsymptotic.lean index cc6cfac79e..6c3cf6c86e 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sigmoid/SiluAsymptotic.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sigmoid/SiluAsymptotic.lean @@ -27,7 +27,7 @@ open Real -- Status: proved /-- Sigmoid is dominated by `exp`: `σ(x) < exp(x)` for all x, because `exp(x) · (1 + exp(-x)) = exp(x) + 1 > 1`. -/ -theorem sigmoid_lt_exp (x : ℝ) : sigmoid x < Real.exp x := by +private theorem sigmoid_lt_exp (x : ℝ) : sigmoid x < Real.exp x := by unfold sigmoid have hpos : (0:ℝ) < 1 + Real.exp (-x) := by linarith [Real.exp_pos (-x)] rw [div_lt_iff₀ hpos] diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spgemm.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spgemm.lean index 4df050d2ba..eb400f2592 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spgemm.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spgemm.lean @@ -1,3 +1,4 @@ +import Mathlib.Data.Real.Basic import Mathlib.Data.Matrix.Basic /-! diff --git a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spmm.lean b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spmm.lean index 264cdf1cb2..d2b8fb4346 100644 --- a/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spmm.lean +++ b/crates/aprender-contracts-staging/lean/ProvableContracts/Theorems/Sparse/Spmm.lean @@ -1,3 +1,4 @@ +import Mathlib.Data.Real.Basic import Mathlib.Data.Matrix.Basic /-! diff --git a/crates/aprender-contracts/Cargo.toml b/crates/aprender-contracts/Cargo.toml index d23b79a747..eb476f33ab 100644 --- a/crates/aprender-contracts/Cargo.toml +++ b/crates/aprender-contracts/Cargo.toml @@ -11,6 +11,13 @@ keywords = ["verification", "formal-methods", "kani", "contracts", "ml-kernels"] categories = ["development-tools::testing", "mathematics"] exclude = ["/tests/", "benches/"] +[package.metadata.kani] +# kani_proofs_cd.rs uses #[kani::stub], which needs -Z stubbing. Kani reads this +# only from the crate directory (`cd crates/aprender-contracts && cargo kani +# --lib`); `cargo kani -p` from the workspace root still needs `-Z stubbing` +# (#3140). +unstable = { stubbing = true } + [features] default = [] kani = [] diff --git a/crates/aprender-contracts/build.rs b/crates/aprender-contracts/build.rs index cae449f0f7..69a8e70a05 100644 --- a/crates/aprender-contracts/build.rs +++ b/crates/aprender-contracts/build.rs @@ -3,6 +3,9 @@ use serde::Deserialize; use std::collections::BTreeMap; use std::path::{Path, PathBuf}; +// The one key builder, shared with `build_helper::env_key` (#4369). +include!("src/env_key.rs"); + #[derive(Deserialize)] struct BindingFile { #[allow(dead_code)] @@ -78,13 +81,7 @@ fn load_binding_file(binding_path: &Path) -> Option { fn emit_binding_vars(bindings: &BindingFile, total: &mut u32, implemented: &mut u32) { for b in &bindings.bindings { - let stem = b - .contract - .trim_end_matches(".yaml") - .to_uppercase() - .replace('-', "_"); - let eq = b.equation.to_uppercase().replace('-', "_"); - let var = format!("CONTRACT_{stem}_{eq}"); + let var = env_key(b.contract.trim_end_matches(".yaml"), &b.equation); println!("cargo:rustc-env={var}={}", b.status); *total += 1; if b.status == "implemented" { @@ -134,11 +131,7 @@ fn emit_contract_file_pre_post(path: &PathBuf, total_pre: &mut usize, total_post return; }; for (eq_name, eq) in &parsed.equations { - let key = format!( - "CONTRACT_{}_{}", - stem, - eq_name.to_uppercase().replace('-', "_") - ); + let key = env_key(&stem, eq_name); emit_pre_post_lists(&key, eq, total_pre, total_post); } } diff --git a/crates/aprender-contracts/src/build_helper.rs b/crates/aprender-contracts/src/build_helper.rs index d42e75731d..80b053ac82 100644 --- a/crates/aprender-contracts/src/build_helper.rs +++ b/crates/aprender-contracts/src/build_helper.rs @@ -98,7 +98,7 @@ pub fn verify_bindings(binding_yaml_path: &str, policy: BindingPolicy) -> Verify }; for binding in ®istry.bindings { - let env_key = make_env_key(&binding.contract, &binding.equation); + let env_key = env_key(&binding.contract, &binding.equation); match binding.status { ImplStatus::Implemented => { @@ -321,14 +321,8 @@ fn scan_source_fns(dir: &Path, found: &mut std::collections::HashSet) { } } -/// Generate the env var key from contract name and equation name. -/// -/// Same convention as `provable-contracts-macros::make_env_key`. -fn make_env_key(contract: &str, equation: &str) -> String { - let contract_part = contract.to_uppercase().replace(['-', '.'], "_"); - let equation_part = equation.to_uppercase().replace(['-', '.'], "_"); - format!("CONTRACT_{contract_part}_{equation_part}") -} +// One definition, shared with this crate's own build.rs (#4369). +include!("env_key.rs"); #[cfg(test)] mod tests { @@ -448,14 +442,39 @@ bindings: // ── make_env_key tests ── + /// Parity with the key the macro READS, over a case table, so the + /// producer's derivation and the consumer's cannot drift (#2699 §4). + #[test] + fn env_key_equals_the_key_the_contract_macro_reads() { + use provable_contracts_macros::contract_env_key; + let table = [ + ( + env_key("rmsnorm-kernel-v1", "rmsnorm"), + contract_env_key!("rmsnorm-kernel-v1", "rmsnorm"), + ), + ( + env_key("attention-kernel-v1", "scaled_dot_product"), + contract_env_key!("attention-kernel-v1", "scaled_dot_product"), + ), + (env_key("v1.0", "eq.1"), contract_env_key!("v1.0", "eq.1")), + ( + env_key("mixed-Case.v2", "Eq-x.y"), + contract_env_key!("mixed-Case.v2", "Eq-x.y"), + ), + ]; + for (producer, consumer) in table { + assert_eq!(producer, consumer); + } + } + #[test] fn test_make_env_key_matches_macro_convention() { assert_eq!( - make_env_key("rmsnorm-kernel-v1", "rmsnorm"), + env_key("rmsnorm-kernel-v1", "rmsnorm"), "CONTRACT_RMSNORM_KERNEL_V1_RMSNORM" ); assert_eq!( - make_env_key("gated-delta-net-v1", "decay"), + env_key("gated-delta-net-v1", "decay"), "CONTRACT_GATED_DELTA_NET_V1_DECAY" ); } @@ -463,7 +482,7 @@ bindings: #[test] fn make_env_key_with_yaml_extension() { assert_eq!( - make_env_key("softmax-kernel-v1.yaml", "softmax"), + env_key("softmax-kernel-v1.yaml", "softmax"), "CONTRACT_SOFTMAX_KERNEL_V1_YAML_SOFTMAX" ); } @@ -471,7 +490,7 @@ bindings: #[test] fn make_env_key_dots_replaced() { assert_eq!( - make_env_key("my.contract.v1", "eq.1"), + env_key("my.contract.v1", "eq.1"), "CONTRACT_MY_CONTRACT_V1_EQ_1" ); } @@ -1226,4 +1245,142 @@ bindings: // A bare filename with no directory separators verify_bindings("nonexistent.yaml", BindingPolicy::WarnOnGaps); } + + // ── #4369: no build.rs hand-rolls the key ── + + /// A line of a build.rs that builds a `CONTRACT__` key itself + /// instead of calling `env_key`: a format string that interpolates + /// straight after `CONTRACT_`, or a `"CONTRACT_"` prefix to concatenate + /// onto. Fixed names (`CONTRACT_BINDING_SOURCE=none`) and comments pass. + fn hand_rolls_contract_key(line: &str) -> bool { + let code = line.trim_start(); + !code.starts_with("//") && (code.contains("CONTRACT_{") || code.contains("\"CONTRACT_\"")) + } + + #[test] + fn hand_rolled_key_detector_case_table() { + let must_match = [ + r#"let var = format!("CONTRACT_{stem}_{eq}");"#, + r#" "CONTRACT_{}_{}","#, + r#"let k = format!("CONTRACT_{su}_{eu}");"#, + r#"let k = "CONTRACT_".to_string() + &stem;"#, + r#"println!("cargo:rustc-env=CONTRACT_{stem}_{eq}={}", s);"#, + ]; + let must_not_match = [ + r#"println!("cargo:rustc-env=CONTRACT_BINDING_SOURCE=none");"#, + r#"println!("cargo:rustc-env=CONTRACT_TOTAL={total}");"#, + r#"let var = provable_contracts::build_helper::env_key(c, e);"#, + r#"println!("cargo:rustc-env={k}_PRE_COUNT={}", n);"#, + r#"// CONTRACT_{STEM}_{EQ}="#, + r#"/// `"x.yaml"` → `"CONTRACT_{}_{}"`"#, + ]; + for l in must_match { + assert!(hand_rolls_contract_key(l), "must match: {l}"); + } + for l in must_not_match { + assert!(!hand_rolls_contract_key(l), "must not match: {l}"); + } + } + + /// Every build.rs in the workspace builds its keys with `env_key` (#4369). + /// A hand-rolled copy that drifts from the macro's format binds nothing, + /// silently (#2699 §4). Outside the workspace (a packaged crate) there is + /// no tree to scan and the test says so rather than passing on zero files. + #[test] + fn no_build_rs_hand_rolls_the_contract_key() { + let crates = Path::new(env!("CARGO_MANIFEST_DIR")).join(".."); + let root = crates.join(".."); + if !root.join("Cargo.toml").exists() || !crates.join("aprender-core").exists() { + eprintln!("SKIP: not in the aprender workspace (packaged crate)"); + return; + } + let mut scanned = Vec::new(); + let mut offenders = Vec::new(); + let mut files = vec![root.join("build.rs")]; + for e in std::fs::read_dir(&crates).expect("read crates/") { + files.push(e.expect("dir entry").path().join("build.rs")); + } + for f in files.into_iter().filter(|f| f.is_file()) { + let src = std::fs::read_to_string(&f).expect("read build.rs"); + for (i, l) in src.lines().enumerate() { + if hand_rolls_contract_key(l) { + offenders.push(format!("{}:{}: {}", f.display(), i + 1, l.trim())); + } + } + scanned.push(f); + } + // 12 producers existed when this guard was written; a scan of fewer + // build.rs files than that is looking in the wrong place. + assert!( + scanned.len() >= 12, + "scanned only {} build.rs files", + scanned.len() + ); + assert!( + offenders.is_empty(), + "build.rs hand-rolls a CONTRACT_* key; call provable_contracts::build_helper::env_key \ + (after trimming \".yaml\"):\n{}", + offenders.join("\n") + ); + } + + /// Every producer reads its binding from THIS repo's `contracts//` + /// (#4369). The old `../../../provable-contracts/contracts//` path + /// resolved only on a checkout with the pre-monorepo sibling beside it, so + /// worktrees and CI built with `CONTRACT_BINDING_SOURCE=none` and nothing + /// was enforced, while the one main checkout enforced a stale copy. + #[test] + fn producer_bindings_are_in_tree() { + let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); + if !root.join("crates/aprender-core").exists() { + eprintln!("SKIP: not in the aprender workspace (packaged crate)"); + return; + } + let producers = [ + ("aprender-compute", "trueno"), + ("aprender-core", "aprender"), + ("aprender-db", "trueno-db"), + ("aprender-orchestrate", "batuta"), + ("aprender-present-cli", "presentar"), + ("aprender-present-core", "presentar"), + ("aprender-rag", "trueno-rag"), + ("aprender-serve", "realizar"), + ("aprender-simulate", "simular"), + ("aprender-train", "entrenar"), + ]; + let mut bad = Vec::new(); + for (krate, name) in producers { + let build = root.join("crates").join(krate).join("build.rs"); + let src = std::fs::read_to_string(&build).expect("read producer build.rs"); + let want = format!("\"../../contracts/{name}/binding.yaml\""); + if !src.contains(&want) { + bad.push(format!("{krate}/build.rs does not read {want}")); + } + // Per statement: a sibling path joined onto binding.yaml. Other + // sibling reads (serve's arch-requirements codegen) are out of scope. + let sibling_binding = |st: &str| { + (st.contains("join(\"provable-contracts\")") + || st.contains("provable-contracts/contracts/")) + && st.contains("binding.yaml") + }; + if src.split(';').any(sibling_binding) { + bad.push(format!( + "{krate}/build.rs still reads binding.yaml from the sibling provable-contracts" + )); + } + if !root + .join("contracts") + .join(name) + .join("binding.yaml") + .is_file() + { + bad.push(format!("contracts/{name}/binding.yaml is missing")); + } + } + assert!( + bad.is_empty(), + "producer bindings not in-tree:\n{}", + bad.join("\n") + ); + } } diff --git a/crates/aprender-contracts/src/env_key.rs b/crates/aprender-contracts/src/env_key.rs new file mode 100644 index 0000000000..bc38586f53 --- /dev/null +++ b/crates/aprender-contracts/src/env_key.rs @@ -0,0 +1,16 @@ +// `include!`d by `build_helper.rs` (which carries the equality test against the +// macro) and by this crate's own build.rs, which cannot build-depend on the +// crate it builds (#4369). +/// The env var key `#[contract(contract, equation = equation)]` reads. +/// +/// A producer (a consuming crate's `build.rs`) must emit its +/// `cargo:rustc-env` vars under exactly this key, plus `_PRE_COUNT`, +/// `_PRE_`, `_POST_COUNT` and `_POST_`. Call this instead of +/// re-deriving the format: a hand-rolled copy that drifts makes every +/// condition land under a name the macro never reads (#2699 §4). It is +/// tested equal to the macro's own `contract_env_key!`. +pub fn env_key(contract: &str, equation: &str) -> String { + let contract_part = contract.to_uppercase().replace(['-', '.'], "_"); + let equation_part = equation.to_uppercase().replace(['-', '.'], "_"); + format!("CONTRACT_{contract_part}_{equation_part}") +} diff --git a/crates/aprender-contracts/src/kernels/kani_proofs.rs b/crates/aprender-contracts/src/kernels/kani_proofs.rs index ec2a087990..54599ccc46 100644 --- a/crates/aprender-contracts/src/kernels/kani_proofs.rs +++ b/crates/aprender-contracts/src/kernels/kani_proofs.rs @@ -32,6 +32,19 @@ use super::swiglu; // Float transcendental stubs // ════════════════════════════════════════════════════════════════════════════ +/// Magnitude bound for harness inputs that feed products, sums or exponents. +/// +/// "Finite for every finite input" is false for these kernels: Kani finds +/// inputs near `f32::MAX` whose dot products overflow to inf, and inf - inf +/// is NaN (#3140). Real activations sit far inside this bound, so a harness +/// states its precondition as `|x| <= ACT_BOUND` instead of `x.is_finite()`. +const ACT_BOUND: f32 = 1.0e4; + +/// `x` is finite and within [`ACT_BOUND`]. +fn act_bounded(x: f32) -> bool { + x.is_finite() && x.abs() <= ACT_BOUND +} + /// Stub for `f32::exp` — models exp(x) for softmax after max-subtraction. /// /// Softmax calls exp(x_i - max(x)), where x_i - max(x) ∈ (-∞, 0]. @@ -42,7 +55,13 @@ use super::swiglu; /// - Lean proves exp(x) > 0 for all x ∈ ℝ (Real.exp_pos) /// - Lean proves exp(0) = 1 (Real.exp_zero) /// - The upper bound 1.0 follows from x_i - max(x) ≤ 0 → exp(·) ≤ 1 -fn stub_exp(_x: f32) -> f32 { +fn stub_exp(x: f32) -> f32 { + // exp(0) = 1 exactly (the max element). Without this the stub let every + // element be tiny at once, so 1/sum overflowed and softmax/attention + // harnesses reported non-finite outputs no real exp can produce (#3140). + if x == 0.0 { + return 1.0; + } let r: f32 = kani::any(); // exp(x - max(x)) for x - max(x) ∈ [-88, 0] gives values in [exp(-88), 1]. // exp(-88) ≈ 6e-39 > f32::MIN_POSITIVE (1.175e-38 is normal). @@ -51,10 +70,28 @@ fn stub_exp(_x: f32) -> f32 { r } +/// Deterministic exp stand-in for EQUIVALENCE harnesses (PMAT-3140). +/// +/// `stub_exp` draws a fresh `kani::any()` per call, so two code paths that each +/// call `exp` on the same argument see two unrelated values and an equivalence +/// assertion between them is refuted by the stub, not by the code. This stub is a +/// pure function of `x` (positive, like exp), so both paths see the same value +/// and the harness judges only the arithmetic around the call. It has no branch +/// and no division: a stub with both timed out at 600 s on 4 elements. +fn stub_exp_det(x: f32) -> f32 { + 1.0 + x * x +} + /// Stub for `f32::sqrt` — returns an arbitrary non-negative finite value. -fn stub_sqrt(_x: f32) -> f32 { +fn stub_sqrt(x: f32) -> f32 { let r: f32 = kani::any(); kani::assume(r >= 0.0 && r.is_finite()); + // For x >= 0, sqrt(x) lies between x and 1 (both representable, so the + // bracket survives rounding). Without it the stub could return 0 for + // sqrt(d_k) and divide the attention scores by zero (#3140). + if x >= 0.0 { + kani::assume(r >= x.min(1.0) && r <= x.max(1.0)); + } r } diff --git a/crates/aprender-contracts/src/kernels/kani_proofs_ab.rs b/crates/aprender-contracts/src/kernels/kani_proofs_ab.rs index 0377fecad8..4bb3ce499d 100644 --- a/crates/aprender-contracts/src/kernels/kani_proofs_ab.rs +++ b/crates/aprender-contracts/src/kernels/kani_proofs_ab.rs @@ -190,18 +190,29 @@ fn verify_softmax_bounded() { } } -/// KANI-RN-001: RMSNorm output is finite when eps > 0. +/// KANI-RN-001: RMSNorm output is finite when eps > 0, |x| <= 1e18 and |gamma| <= 1e30. /// Obligation: RN-INV-001 /// Strategy: exhaustive /// Bound: 16 elements +/// +/// Bounds (PMAT-3140). Input: unbounded, Kani 0.67 reported +/// `rmsnorm_scalar.NaN.5 FAILURE "NaN on division"` at `1.0 / rms` (rmsnorm.rs:37) +/// on a trace with |x| ~ 1.9e36, where sum(x^2) overflows. IEEE gives +/// sqrt(inf) = inf and 1/inf = 0, so this is most likely CBMC's builtin sqrtf +/// model on inf, not a runtime NaN. The bound stays anyway: past it the real +/// output collapses to all zeros, which is finite but is not RMSNorm. Gamma: pure +/// arithmetic, not a Kani finding. The normalized value reaches sqrt(n) = 4, so +/// |gamma| > f32::MAX / 4 ~ 8.5e37 overflows, and 1e30 is conservative. #[kani::proof] #[kani::unwind(17)] fn verify_rmsnorm_finiteness() { const N: usize = 16; let input: [f32; N] = kani::any(); let gamma: [f32; N] = kani::any(); - kani::assume(input.iter().all(|x| x.is_finite())); - kani::assume(gamma.iter().all(|x| x.is_finite())); + // |x| <= 1e18 keeps sum(x^2) <= 1.6e37 < f32::MAX. Past that, sum_sq overflows, + // inv_rms becomes 0 and every output is 0: finite, but no longer RMSNorm. + kani::assume(input.iter().all(|x| x.is_finite() && x.abs() <= 1.0e18)); + kani::assume(gamma.iter().all(|x| x.is_finite() && x.abs() <= 1.0e30)); let eps: f32 = kani::any(); kani::assume(eps > 0.0 && eps.is_finite() && eps < 1.0); @@ -222,7 +233,12 @@ fn verify_rmsnorm_finiteness() { /// Obligation: RN-INV-002 /// Strategy: exhaustive /// Bound: 16 elements -/// Inlines the RMS computation to check the intermediate value. +/// Inlines the RMS computation to check the intermediate value, then binds the +/// property to the KERNEL: the inline copy alone never calls `rmsnorm_scalar`, +/// so a kernel that dropped or negated `+ eps` stayed green (#3140 finding). +/// The all-zero input is the worst case (sum_sq = 0, denominator = eps): there +/// the kernel's output is finite iff its denominator was positive, since a zero +/// denominator gives 0 * inf = NaN and a negative one gives sqrt(<0) = NaN. #[kani::proof] #[kani::unwind(17)] fn verify_rms_positive() { @@ -241,6 +257,20 @@ fn verify_rms_positive() { let denom = sum_sq / N as f32 + eps; assert!(denom > 0.0, "KANI-RN-002: denominator = {} <= 0", denom); + + let zeros = [0.0f32; N]; + let gamma: [f32; N] = kani::any(); + kani::assume(gamma.iter().all(|g| g.is_finite())); + let mut output = [1.0f32; N]; + rmsnorm::rmsnorm_scalar(&zeros, &gamma, eps, &mut output); + for i in 0..N { + assert!( + output[i] == 0.0, + "KANI-RN-002: kernel output[{}] = {} on zero input (denominator not positive)", + i, + output[i] + ); + } } /// KANI-LN-001: LayerNorm output has zero mean (with gamma=1, beta=0). diff --git a/crates/aprender-contracts/src/kernels/kani_proofs_cd.rs b/crates/aprender-contracts/src/kernels/kani_proofs_cd.rs index ff9106cfa0..78033c7f51 100644 --- a/crates/aprender-contracts/src/kernels/kani_proofs_cd.rs +++ b/crates/aprender-contracts/src/kernels/kani_proofs_cd.rs @@ -59,12 +59,15 @@ fn verify_swiglu_zero_preservation() { /// KANI-SG-002: Fused SwiGLU equivalence: swiglu(gate, value) = silu(gate) * value. /// Obligation: SG-INV-002 /// Strategy: stub_float -/// Bound: 4 elements +/// Bound: 2 elements (elements are independent; 4 timed out at 600 s bit-exact) +/// +/// Not in scripts/kani_parity_chain.sh: proving two symbolic f32 division circuits +/// bit-identical timed out at 600 s (2 and 4 elements). FALSIFY-SG-002 covers it. #[kani::proof] -#[kani::unwind(5)] -#[kani::stub(f32::exp, stub_exp)] +#[kani::unwind(3)] +#[kani::stub(f32::exp, stub_exp_det)] fn verify_swiglu_fused_equivalence() { - const N: usize = 4; + const N: usize = 2; let gate: [f32; N] = kani::any(); let value: [f32; N] = kani::any(); kani::assume(gate.iter().all(|x| x.is_finite())); @@ -84,8 +87,10 @@ fn verify_swiglu_fused_equivalence() { for i in 0..N { assert!( - (fused[i] - unfused[i]).abs() < 1e-5 - || (!fused[i].is_finite() && !unfused[i].is_finite()), + // Same formula, same exp value (deterministic stub): bitwise equal. + // An absolute 1e-5 tolerance is meaningless at |value| ~ 1e34. + fused[i].to_bits() == unfused[i].to_bits() + || (fused[i].is_nan() && unfused[i].is_nan()), "KANI-SG-002: fused[{}] = {} != unfused = {}", i, fused[i], @@ -156,7 +161,7 @@ fn verify_cross_entropy_non_negative() { fn verify_log_softmax_upper_bound() { const N: usize = 8; let logits: [f32; N] = kani::any(); - kani::assume(logits.iter().all(|x| x.is_finite())); + kani::assume(logits.iter().all(|x| act_bounded(*x))); let mut output = [0.0f32; N]; cross_entropy::log_softmax_scalar(&logits, &mut output); @@ -203,7 +208,7 @@ fn verify_cross_entropy_finite() { fn verify_rope_norm_preservation() { const D: usize = 4; let x: [f32; D] = kani::any(); - kani::assume(x.iter().all(|v| v.is_finite())); + kani::assume(x.iter().all(|v| act_bounded(*v))); let position: u32 = kani::any(); kani::assume(position < 1024); @@ -280,9 +285,9 @@ fn verify_attention_weights_normalize() { let q: [f32; N * DK] = kani::any(); let k: [f32; M * DK] = kani::any(); let v: [f32; M * DV] = kani::any(); - kani::assume(q.iter().all(|x| x.is_finite())); - kani::assume(k.iter().all(|x| x.is_finite())); - kani::assume(v.iter().all(|x| x.is_finite())); + kani::assume(q.iter().all(|x| act_bounded(*x))); + kani::assume(k.iter().all(|x| act_bounded(*x))); + kani::assume(v.iter().all(|x| act_bounded(*x))); let mut output = [0.0f32; N * DV]; attention::attention_scalar(&q, &k, &v, N, M, DK, DV, &mut output); @@ -301,7 +306,7 @@ fn verify_attention_weights_normalize() { /// Strategy: stub_float /// Bound: seq_len=2, d_k=2, d_v=2, 2 heads, 2 kv_heads #[kani::proof] -#[kani::unwind(5)] +#[kani::unwind(9)] #[kani::stub(f32::exp, stub_exp)] #[kani::stub(f32::sqrt, stub_sqrt)] fn verify_gqa_weight_normalization() { @@ -314,9 +319,9 @@ fn verify_gqa_weight_normalization() { let q: [f32; HEADS * SEQ * DK] = kani::any(); let k: [f32; KV_HEADS * SEQ * DK] = kani::any(); let v: [f32; KV_HEADS * SEQ * DV] = kani::any(); - kani::assume(q.iter().all(|x| x.is_finite())); - kani::assume(k.iter().all(|x| x.is_finite())); - kani::assume(v.iter().all(|x| x.is_finite())); + kani::assume(q.iter().all(|x| act_bounded(*x))); + kani::assume(k.iter().all(|x| act_bounded(*x))); + kani::assume(v.iter().all(|x| act_bounded(*x))); let mut output = [0.0f32; HEADS * SEQ * DV]; gqa::gqa_scalar(&q, &k, &v, SEQ, DK, DV, HEADS, KV_HEADS, &mut output); @@ -335,7 +340,7 @@ fn verify_gqa_weight_normalization() { /// Strategy: stub_float /// Bound: seq_len=2, d_k=2, d_v=2, 2 heads, 2 kv_heads #[kani::proof] -#[kani::unwind(5)] +#[kani::unwind(9)] #[kani::stub(f32::exp, stub_exp)] #[kani::stub(f32::sqrt, stub_sqrt)] fn verify_gqa_mha_equivalence() { @@ -347,9 +352,9 @@ fn verify_gqa_mha_equivalence() { let q: [f32; HEADS * SEQ * DK] = kani::any(); let k: [f32; HEADS * SEQ * DK] = kani::any(); let v: [f32; HEADS * SEQ * DV] = kani::any(); - kani::assume(q.iter().all(|x| x.is_finite())); - kani::assume(k.iter().all(|x| x.is_finite())); - kani::assume(v.iter().all(|x| x.is_finite())); + kani::assume(q.iter().all(|x| act_bounded(*x))); + kani::assume(k.iter().all(|x| act_bounded(*x))); + kani::assume(v.iter().all(|x| act_bounded(*x))); // GQA with kv_heads = num_heads is standard MHA let mut output_gqa = [0.0f32; HEADS * SEQ * DV]; @@ -372,7 +377,7 @@ fn verify_gqa_mha_equivalence() { /// Strategy: stub_float /// Bound: seq_len=2, d_k=2, d_v=2, 2 heads, 2 kv_heads #[kani::proof] -#[kani::unwind(5)] +#[kani::unwind(9)] #[kani::stub(f32::exp, stub_exp)] #[kani::stub(f32::sqrt, stub_sqrt)] fn verify_gqa_convex_bound() { @@ -385,9 +390,9 @@ fn verify_gqa_convex_bound() { let q: [f32; HEADS * SEQ * DK] = kani::any(); let k: [f32; KV_HEADS * SEQ * DK] = kani::any(); let v: [f32; KV_HEADS * SEQ * DV] = kani::any(); - kani::assume(q.iter().all(|x| x.is_finite())); - kani::assume(k.iter().all(|x| x.is_finite())); - kani::assume(v.iter().all(|x| x.is_finite())); + kani::assume(q.iter().all(|x| act_bounded(*x))); + kani::assume(k.iter().all(|x| act_bounded(*x))); + kani::assume(v.iter().all(|x| act_bounded(*x))); let mut output = [0.0f32; HEADS * SEQ * DV]; gqa::gqa_scalar(&q, &k, &v, SEQ, DK, DV, HEADS, KV_HEADS, &mut output); @@ -408,7 +413,8 @@ fn verify_gqa_convex_bound() { /// Strategy: stub_float /// Bound: n=4, d=2, tile_size=2 #[kani::proof] -#[kani::unwind(5)] +// 9, not 5: the `iter().all` finiteness assumptions walk N*D = 8 elements. +#[kani::unwind(9)] #[kani::stub(f32::exp, stub_exp)] fn verify_online_softmax_2tiles() { const N: usize = 4; @@ -417,9 +423,11 @@ fn verify_online_softmax_2tiles() { let q: [f32; N * D] = kani::any(); let k: [f32; N * D] = kani::any(); let v: [f32; N * D] = kani::any(); - kani::assume(q.iter().all(|x| x.is_finite())); - kani::assume(k.iter().all(|x| x.is_finite())); - kani::assume(v.iter().all(|x| x.is_finite())); + // |x| <= 1e15: a D=2 dot product stays <= 2e30. Unbounded finite q/k overflow the + // score to inf, and inf - inf in the running max is NaN (Kani found this, PMAT-3140). + kani::assume(q.iter().all(|x| x.is_finite() && x.abs() <= 1.0e15)); + kani::assume(k.iter().all(|x| x.is_finite() && x.abs() <= 1.0e15)); + kani::assume(v.iter().all(|x| x.is_finite() && x.abs() <= 1.0e15)); let mut output = [0.0f32; N * D]; flash_attention::flash_attention_scalar(&q, &k, &v, N, D, 2, &mut output); diff --git a/crates/aprender-contracts/src/lib.rs b/crates/aprender-contracts/src/lib.rs index 6ddd406061..5ecbe14749 100644 --- a/crates/aprender-contracts/src/lib.rs +++ b/crates/aprender-contracts/src/lib.rs @@ -68,3 +68,4 @@ pub mod schema; pub mod scoring; pub mod tla_gen; pub mod traits; +pub mod tree; diff --git a/crates/aprender-contracts/src/lint/capability_cells_gate.rs b/crates/aprender-contracts/src/lint/capability_cells_gate.rs new file mode 100644 index 0000000000..2d188b8f0f --- /dev/null +++ b/crates/aprender-contracts/src/lint/capability_cells_gate.rs @@ -0,0 +1,297 @@ +//! ONT-4c5 inside the `shapes` gate: the validator's half of `capability-cells`. +//! +//! The extractor half ([`crate::ontology::capability_cells::apply`]) writes what it finds and the set difference; +//! this module decides what the gate does with it: +//! +//! - **Corpus** — [`corpus`] runs `apply` on the gate's own copy of the graph, only when a contract declares the +//! shape. A receipt version that is not dotted numerals is the declaration's fault (exit 3); |D| = 0 is R-2's +//! decline ([`super::shapes_gate::ShapesOutcome::EmptyDomain`], exit 2), never Pass and never RED. +//! - **Positive control** — [`control`] runs the SAME `apply` over [`PLANT`] (a required rung with no receipt) and +//! then the ARMED `capability-cells` shape over the result. Both must refuse the plant's one cell, or +//! `pc_shapes["capability-cells"]` is `not-fired` and the gate declines: a plant that checked only the Rust set +//! difference would stay `fired` with the shape deleted or disarmed (plan grill round 1). +//! - **Wiring** — [`wiring_holds`]: the rungs the shape flagged on the corpus must be exactly the rungs that own a +//! `not_run` cell. A corpus that bypassed `apply`, or a shape that stopped reading `model:notRunCell`, disagrees +//! and the gate declines (`Differential`) — at run time, not only in a test (plan grill round 2). +//! - **Names** — [`findings`]: one finding per NotRun cell, naming it, because a SHACL `maxCount 0` result names +//! the rung, not which of its hosts is missing. + +use std::collections::{BTreeMap, BTreeSet}; + +use crate::ontology::arming::ArmedShapes; +use crate::ontology::capability_cells::{self, CapabilityCells, CellsError}; +use crate::ontology::extract::gguf::{self, Rung}; +use crate::ontology::rdf::{iri, Graph}; +use crate::ontology::receipts::Receipt; +use crate::ontology::shapes::{self, NodeShape, Report, Severity}; + +use super::finding::LintFinding; +use super::rules::RuleSeverity; + +/// The shape's id, as contracts/ont-capability-cells-v1.yaml declares it and `armed_shapes` names it. +pub const SHAPE: &str = "capability-cells"; + +/// The positive control. Byte-identical to the tracked `tests/fixtures/ont/capability-cells-plant.yaml` (a test +/// holds the two together): a crate published without the workspace's `tests/` still carries its plant. +pub const PLANT: &str = include_str!("capability_cells_plant.yaml"); + +/// The one cell the plant must be refused for. +pub const PLANT_CELL: &str = "capability-cells-plant@plant-host"; + +/// What `pv lint --gate shapes --format json` reports as `capability_cells`. +#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)] +pub struct CapabilityCellsReport { + /// The release the cells were measured at. + pub v_star: Option, + /// D, sorted `@` ids. + pub domain: Vec, + /// `D \ {Pass | Fail}`, sorted. Non-empty ⇒ the armed shape fails. + pub not_run: Vec, + /// Row labels `Verdict::from_label` does not know, by file and rung. + pub refused_labels: Vec, +} + +impl From<&CapabilityCells> for CapabilityCellsReport { + fn from(cc: &CapabilityCells) -> Self { + Self { + v_star: cc.v_star.clone(), + domain: cc.domain.iter().cloned().collect(), + not_run: cc.not_run.iter().cloned().collect(), + refused_labels: cc.refused_labels.clone(), + } + } +} + +/// Does any contract declare the shape? Without it there is nothing to compute and nothing to control. +#[must_use] +pub fn declared(shapes: &[NodeShape]) -> bool { + shapes.iter().any(|s| s.id == SHAPE) +} + +/// Run `apply` on the corpus graph. `Ok(None)` when the shape is not declared. +pub fn corpus( + graph: &mut Graph, + shapes: &[NodeShape], + rungs: &[Rung], + receipts: &[Receipt], +) -> Result, CellsError> { + if !declared(shapes) { + return Ok(None); + } + capability_cells::apply(graph, rungs, receipts).map(Some) +} + +/// `fired` iff the plant's one cell is NotRun AND the ARMED `capability-cells` shape refuses the plant's node. +#[must_use] +pub fn control(shapes: &[NodeShape], arming: &ArmedShapes) -> String { + let fired = plant_refused(shapes, arming); + if fired { "fired" } else { "not-fired" }.to_string() +} + +fn plant_refused(shapes: &[NodeShape], arming: &ArmedShapes) -> bool { + if !arming.is_armed(SHAPE) { + return false; + } + let Some(shape) = shapes.iter().find(|s| s.id == SHAPE) else { + return false; + }; + let Ok(doc) = serde_yaml::from_str::(PLANT) else { + return false; + }; + let rungs = gguf::rungs_of("capability-cells-plant", &doc); + let mut g = Graph::new(); + for r in &rungs { + gguf::emit_rung(&mut g, r); + } + let Ok(cc) = capability_cells::apply(&mut g, &rungs, &[]) else { + return false; + }; + if cc.not_run.iter().map(String::as_str).ne([PLANT_CELL]) { + return false; + } + let report = shapes::validate(&g, std::slice::from_ref(shape)); + report + .results + .iter() + .any(|r| r.shape == SHAPE && r.severity == Severity::Violation) +} + +/// The NODES the shape flagged on the corpus are exactly the nodes of the rungs that own a `not_run` cell. +/// Compared by node, not by rung id: ONT-4c1 keys a rung's node by sha256, so two rungs sharing a sha share one +/// node, and an id-level comparison would read the healthy twin as flagged and decline an honest tree (review +/// lane C, round 2). +#[must_use] +pub fn wiring_holds(report: &Report, rungs: &[Rung], cc: &CapabilityCells) -> bool { + let flagged: BTreeSet<&str> = report + .results + .iter() + .filter(|r| r.shape == SHAPE && r.severity == Severity::Violation) + .map(|r| r.focus.as_str()) + .collect(); + // a refused label makes its row NotRun, so its rung already owns a not_run cell + let owning_ids: BTreeSet<&str> = cc + .not_run + .iter() + .filter_map(|id| id.rsplit_once('@').map(|(rung, _)| rung)) + .collect(); + let owning: BTreeSet = rungs + .iter() + .filter(|r| r.required && owning_ids.contains(r.id.as_str())) + .map(|r| iri("model", &r.sha256)) + .collect(); + flagged == owning.iter().map(String::as_str).collect() +} + +/// One finding per NotRun cell and per refused label, naming it. Error when the shape is armed, Warning when not. +#[must_use] +pub fn findings(cc: &CapabilityCells, arming: &ArmedShapes) -> Vec { + let severity = if arming.is_armed(SHAPE) { + RuleSeverity::Error + } else { + RuleSeverity::Warning + }; + let at = cc.v_star.as_deref().unwrap_or("none"); + let mut out: Vec = cc + .not_run + .iter() + .map(|id| { + let why = cc.cells.get(id).map_or_else( + || "no witness row in the V* receipt".to_string(), + |v| format!("its V* row says {v}"), + ); + LintFinding::new( + "PV-ONT-013", + severity, + format!("capability cell {id} is Unknown(NotRun) at V* {at} ({why}) — a required cell is Pass or Fail, never NotRun (ONT-4c5)"), + "contracts/ont-capability-cells-v1.yaml".to_string(), + ) + }) + .collect(); + out.extend(cc.refused_labels.iter().map(|l| { + LintFinding::new( + "PV-ONT-013", + severity, + format!("row label refused by name: {l} — not a fleet label (verdict.rs FLEET_LABELS)"), + "contracts/ont-capability-cells-v1.yaml".to_string(), + ) + })); + for f in &mut out { + f.contract_stem = Some(SHAPE.to_string()); + } + out +} + +/// `pc_shapes`: one entry per shape that carries its own positive control. Empty when none is declared. +#[must_use] +pub fn pc_shapes(shapes: &[NodeShape], arming: &ArmedShapes) -> BTreeMap { + let mut out = BTreeMap::new(); + if declared(shapes) { + out.insert(SHAPE.to_string(), control(shapes, arming)); + } + out +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn the_embedded_plant_is_the_tracked_fixture_byte_for_byte() { + let tracked = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../tests/fixtures/ont/capability-cells-plant.yaml"); + let text = std::fs::read_to_string(&tracked).expect("tracked plant fixture exists"); + assert_eq!( + PLANT, text, + "tests/fixtures/ont/capability-cells-plant.yaml drifted from the embedded plant" + ); + } + + /// Review lane C: the §5 probe reads `pc_shapes` and `capability_cells` at the TOP level of the single-gate + /// report, which flattens `extra`; so they must be DIRECT keys of the serialized `GateExtra::Shapes`, not nested + /// under `controls` (they sit in a boxed, flattened `ShapesControls`). + fn rung(id: &str, sha: &str) -> Rung { + Rung { + id: id.into(), + sha256: sha.into(), + arch: "qwen2".into(), + gguf: format!("{id}.gguf"), + backends: vec!["cpu".into()], + hosts: vec!["lambda".into()], + required: true, + contract: "ladder".into(), + } + } + + fn shape() -> NodeShape { + let doc: serde_yaml::Value = serde_yaml::from_str( + "shapes:\n - id: capability-cells\n targetClass: model:RequiredModel\n properties:\n - {path: model:notRunCell, maxCount: 0}\n - {path: model:refusedLabel, maxCount: 0}\n", + ) + .expect("yaml"); + shapes::parse_shapes("fixture", &doc) + .expect("in subset") + .into_iter() + .next() + .expect("one shape") + } + + /// Two required rungs with DIFFERENT ids and ONE sha share one node; only one of them is NotRun. The honest + /// wiring must still hold (compared by node), and a corpus that skipped `apply` must not. + #[test] + fn wiring_is_compared_by_node_so_a_shared_sha_does_not_decline_an_honest_tree() { + let sha = "c".repeat(64); + let rungs = [rung("twin-a", &sha), rung("twin-b", &sha)]; + let ok_row = r#"{"id":"twin-b","sha256":"SHA","green":true,"capability_match":{"passed":true,"skipped":false},"backends":{"cpu":{"ran":true,"fallback":false}}}"#.replace("SHA", &sha); + let rec = crate::ontology::receipts::parse( + "r.json", + &format!(r#"{{"schema":"apr-model-ladder-receipt/v2","host":"lambda","version":"0.69.1","sha":"x","cc":"8.9","gpu":"g","rungs":[{ok_row}]}}"#), + ) + .expect("parses"); + let mut g = Graph::new(); + for r in &rungs { + gguf::emit_rung(&mut g, r); + } + let cc = capability_cells::apply(&mut g, &rungs, &[rec]).expect("applies"); + assert_eq!( + cc.not_run.iter().map(String::as_str).collect::>(), + ["twin-a@lambda"] + ); + let report = shapes::validate(&g, &[shape()]); + assert!(wiring_holds(&report, &rungs, &cc), "{:?}", report.results); + // a corpus that bypassed apply: the same cells, but no edges in the validated graph → must NOT hold + let mut bare = Graph::new(); + for r in &rungs { + gguf::emit_rung(&mut bare, r); + } + let bypassed = shapes::validate(&bare, &[shape()]); + assert!(!wiring_holds(&bypassed, &rungs, &cc)); + } + + #[test] + fn pc_shapes_and_capability_cells_serialize_as_direct_keys_of_the_shapes_extra() { + let dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../tests/fixtures/ont/capcells-clean/contracts"); + let outcome = super::super::shapes_gate::run_shapes_gate(&dir); + let super::super::shapes_gate::ShapesOutcome::Ran { result, .. } = outcome else { + panic!("capcells-clean must run: {outcome:?}"); + }; + let v = serde_json::to_value(result.extra.as_ref().expect("extra")).expect("serializes"); + assert_eq!(v["type"], "shapes", "{v}"); + assert!( + v.get("controls").is_none(), + "the box leaked as a nested key: {v}" + ); + assert_eq!(v["pc_shapes"][SHAPE], "fired", "{v}"); + assert!(v["pc_extract"].is_object(), "{v}"); + assert_eq!(v["capability_cells"]["v_star"], "0.69.1", "{v}"); + assert_eq!( + v["capability_cells"]["domain"].as_array().map(Vec::len), + Some(4), + "{v}" + ); + assert_eq!( + v["capability_cells"]["not_run"], + serde_json::json!([]), + "{v}" + ); + } +} diff --git a/crates/aprender-contracts/src/lint/capability_cells_plant.yaml b/crates/aprender-contracts/src/lint/capability_cells_plant.yaml new file mode 100644 index 0000000000..46033e09bb --- /dev/null +++ b/crates/aprender-contracts/src/lint/capability_cells_plant.yaml @@ -0,0 +1,14 @@ +# ONT-4c5 positive control (paiml/infra paiml-ontology.md v4.12 §5 ONT-4c5): a REQUIRED rung with NO receipt. +# The shapes gate runs it every time through the same capability_cells::apply as the corpus, then through the +# ARMED `capability-cells` shape, and must refuse exactly one cell, capability-cells-plant@plant-host. If it +# does not, pc_shapes["capability-cells"] is not-fired and the gate declines. The tracked copy under +# tests/fixtures/ont/ and crates/aprender-contracts/src/lint/ must stay byte-identical (a unit test holds them). +ladder: + rungs: + - id: capability-cells-plant + sha256: "0000000000000000000000000000000000000000000000000000000000000000" + arch: plant + gguf: capability-cells-plant.gguf + backends: [cpu] + hosts: [plant-host] + required: true diff --git a/crates/aprender-contracts/src/lint/mod.rs b/crates/aprender-contracts/src/lint/mod.rs index 88c8c70cad..6add325c69 100644 --- a/crates/aprender-contracts/src/lint/mod.rs +++ b/crates/aprender-contracts/src/lint/mod.rs @@ -9,6 +9,7 @@ //! Spec: `docs/specifications/sub/lint.md` pub mod cache; +pub mod capability_cells_gate; mod composition_gate; pub mod config; pub mod diff; @@ -17,6 +18,7 @@ pub mod finding; mod gates; pub use gates::collect_yaml_files; mod gates_extended; +pub mod ratchet_gates; pub mod relations_gate; pub mod rules; pub mod sarif; @@ -25,6 +27,7 @@ pub mod sigma_gate; pub mod sigma_symbols; mod strict_test_binding; pub mod trend; +pub mod valid_under_gate; use std::collections::{HashMap, HashSet}; use std::path::Path; @@ -137,6 +140,21 @@ pub enum GateDetail { Skipped { reason: String }, } +/// The `shapes` gate's positive controls and ONT-4c5's cell report — one boxed, flattened block of +/// [`GateExtra::Shapes`]. +#[derive(Debug, Clone, Serialize)] +pub struct ShapesControls { + /// The extractor positive controls: `gguf` (corrupt magic refused), `apr-model` (lying header refused). + pub pc_extract: std::collections::BTreeMap, + /// ONT-4c5: per-shape positive controls (`capability-cells` → `fired`); the gate never reaches `Ran` with + /// one that is not. + pub pc_shapes: std::collections::BTreeMap, + /// ONT-4c5: the capability-cell domain D and its NotRun cells at the current release — absent unless a + /// contract declares the `capability-cells` shape. + #[serde(skip_serializing_if = "Option::is_none")] + pub capability_cells: Option, +} + /// Out-of-band structured detail for gates that post-date the frozen [`GateDetail`] /// vocabulary. /// @@ -144,6 +162,23 @@ pub enum GateDetail { /// it and none can `match` it. That is what makes it extensible where `GateDetail` is /// not, and it is `#[non_exhaustive]` from birth so the *next* post-0.3.1 gate does /// not have to repeat this exercise. +// `Shapes` is 304 bytes against a 88-byte second-largest, and adding `declines` to it is what +// crossed the threshold — `origin/main` at 237fbc32f lints clean, this branch does not, so the +// finding is this PR's and not inherited. +// +// ALLOWED RATHER THAN BOXED, deliberately. `large_enum_variant` is a cost heuristic about copying +// and stack size: a `GateExtra` is built ONCE PER GATE RUN, moved a handful of times, and then +// serialised. There is no hot path here for 216 bytes to matter on, so the lint is measuring a cost +// this type does not pay. +// +// The alternatives are worse. Boxing one field (clippy suggests `pc_extract`) reclaims 16 bytes and +// does not clear the ratio, so it would be churn that silences nothing. Boxing the whole payload — +// `Shapes(Box)` — turns a struct variant into a newtype variant, which CHANGES THE +// SERDE REPRESENTATION of a `#[serde(tag = "type")]` enum that downstream consumers parse; the SLK +// gate reads this JSON. Breaking a wire format to satisfy a stack-size heuristic is the wrong trade. +// +// If `GateExtra` ever ends up in a loop or a large collection, this allow is the thing to revisit. +#[allow(clippy::large_enum_variant)] #[derive(Debug, Clone, Serialize)] #[serde(tag = "type")] #[non_exhaustive] @@ -198,6 +233,60 @@ pub enum GateExtra { /// Findings. violations: usize, }, + /// ONT-7: kernel-kind contracts carry a world index (`metadata.valid_under.world`, a key of Σ's `worlds`). + #[serde(rename = "valid_under")] + ValidUnder { + /// The worlds Σ declares — the index a `valid_under.world` must resolve into. + worlds: Vec, + /// Contract files read. + contracts_checked: usize, + /// Of those, kernel-kind and not a registry — the class the row obliges. + kernel_contracts: usize, + /// Contracts (any kind) carrying `metadata.valid_under`. + contracts_with_valid_under: usize, + /// Kernel-kind contracts carrying none — the debt, shrink-only against the baseline. + contracts_without_valid_under: usize, + /// The top-level `contracts_without_valid_under` in `lint-baseline.json`; `None` = not recorded. + baseline: Option, + /// `world=count` over the contracts whose `valid_under` passed every rule. + by_world: Vec, + /// Findings. + violations: usize, + }, + /// PVL-001 EV-11: Lean theorem modules named by a book page — the `unpaired_theorem_modules` ratchet. + #[serde(rename = "theorem_pairing")] + TheoremPairing { + /// The Lean base the modules were read under. + lean_base: String, + /// `.md` pages read under the book roots. + book_pages: usize, + /// `.lean` files under `/ProvableContracts/Theorems/`. + theorem_modules: usize, + /// Of those, named in full by at least one book page. + paired: usize, + /// The debt, shrink-only against the baseline. + unpaired_theorem_modules: usize, + /// The top-level `unpaired_theorem_modules` in `lint-baseline.json`; `None` = not recorded (reported only). + baseline: Option, + /// The unpaired modules, dotted, in byte order. + unpaired: Vec, + /// Findings. + violations: usize, + }, + /// PVL-001 EV-11: kernel-kind contracts with an empty `metadata.depends_on` — the ratchet. + #[serde(rename = "depends_on_present")] + DependsOnPresent { + /// Contract files parsed. + contracts_checked: usize, + /// Of those, kernel-kind and not a registry. + kernel_contracts: usize, + /// Kernel-kind contracts whose `metadata.depends_on` is empty — the debt, shrink-only. + contracts_without_depends_on: usize, + /// The top-level `contracts_without_depends_on` in `lint-baseline.json`; `None` = not recorded (reported only). + baseline: Option, + /// Findings. + violations: usize, + }, /// ONT-4b: the shapes gate — every `shape:` block over the extracted graph, with the plant. #[serde(rename = "shapes")] Shapes { @@ -221,12 +310,36 @@ pub enum GateExtra { armed_shapes: Vec, /// Shapes computed and reported but not armed — their violations are in `unarmed_violations`. not_armed_shapes: Vec, + /// #3610: shapes that graded ZERO focus nodes, named — the reach the gate did not have. + /// + /// A separate list from `not_armed_shapes` on purpose. "Not armed by policy" and "armed and + /// measured nothing" are different facts, and folding the second into the first would file a + /// vacuity as a deliberate choice — which is how the defect hid in the first place. + /// + /// **This carries EVERY shape that graded zero, armed or unarmed**, and it is the only list + /// any of them appears in: a vacuity is in neither `armed_shapes` (the tool's claim about + /// what it MEASURED) nor `not_armed_shapes` (a policy choice it never made). + /// + /// The two differ in what they do to the verdict, not in whether they are listed here. An + /// ARMED vacuity drives the verdict to `Unknown(NoFocus)` and the run declines — unless an + /// armed shape that DID grade something found a violation, in which case the verdict is + /// `Fail` (a measured violation outranks a vacuity, #3622). An UNARMED one leaves the + /// verdict alone, because it never fed it. Both are named, because a reader + /// needs to know the gate looked at nothing for them either way. + /// + /// An earlier draft of this comment said an armed vacuity "does not reach here at all", + /// which was false — nothing returns early at the verdict, and both kinds reach this field. + /// A quorum lane caught the sentence; the code beside it had the matching bug. + declines: Vec, /// Violations from unarmed shapes (named in the findings as warnings; never in the meet). unarmed_violations: usize, /// Focus nodes each extractor produced: `pv-contract`, `gguf`, `apr-model`. by_entity_type: std::collections::BTreeMap, - /// The extractor positive controls: `gguf` (corrupt magic refused), `apr-model` (lying header refused). - pc_extract: std::collections::BTreeMap, + /// The positive controls (`pc_extract`, `pc_shapes`) and ONT-4c5's `capability_cells`, boxed and + /// FLATTENED: the JSON keys stay top-level where the §5 probes read them, and the variant stays under + /// clippy's large-enum-variant bound (it sat exactly at it before ONT-4c5). + #[serde(flatten)] + controls: Box, /// Ladder receipt files read under `evidence/dogfood/models/`. receipts: usize, /// Receipt rows whose `sha256` equals a rung's. @@ -556,6 +669,21 @@ pub fn run_lint(config: &LintConfig) -> LintReport { gates.push(shapes_gate_result); all_findings.append(&mut shapes_findings); + // Gate 13: valid-under (ONT-7). Same R-8 shape: computed in every run, armed per repo. + let (valid_under_gate_result, mut valid_under_findings) = + valid_under_result(config.contract_dir, validation_passed); + gates.push(valid_under_gate_result); + all_findings.append(&mut valid_under_findings); + + // Gates 14, 15: theorem-pairing, depends-on-present (PVL-001 EV-11). Same R-8 shape: computed in every run, + // armed per repo. + for (name, run) in RATCHET_GATES { + let (result, mut findings) = + ratchet_result(config.contract_dir, validation_passed, name, run); + gates.push(result); + all_findings.append(&mut findings); + } + // Gate 9: strict test-binding (Issue #1510, opt-in via --strict-test-binding) if config.strict_test_binding { push_gate( @@ -635,6 +763,10 @@ pub enum NamedGateOutcome { Relations(relations_gate::RelationsOutcome), /// The `shapes` gate (ONT-4b), with four non-verdict answers (unsupported shape, no shapes, no focus, control failed). Shapes(shapes_gate::ShapesOutcome), + /// The `valid-under` gate (ONT-7), with three non-verdict answers (no Σ, malformed Σ, no kernel contract). + ValidUnder(valid_under_gate::ValidUnderOutcome), + /// A PVL-001 EV-11 ratchet (`theorem-pairing`, `depends-on-present`), whose one non-verdict answer is a decline. + Ratchet(ratchet_gates::RatchetOutcome), /// A gate that ran and judged the corpus. Ran { result: Box, @@ -666,6 +798,15 @@ pub fn run_named_gate_with( NamedGateOutcome::Shapes(shapes_gate::run_shapes_gate_with(contract_dir, shapes_opts)) } "sigma" => NamedGateOutcome::Sigma(sigma_gate::run_sigma_gate(contract_dir)), + "valid-under" => { + NamedGateOutcome::ValidUnder(valid_under_gate::run_valid_under_gate(contract_dir)) + } + "theorem-pairing" => { + NamedGateOutcome::Ratchet(ratchet_gates::run_theorem_pairing_gate(contract_dir)) + } + "depends-on-present" => { + NamedGateOutcome::Ratchet(ratchet_gates::run_depends_on_present_gate(contract_dir)) + } "validate" => { let (contracts, parse_errors) = load_contracts(contract_dir); let (result, findings) = run_validate_gate(&contracts, &parse_errors); @@ -679,7 +820,42 @@ pub fn run_named_gate_with( } /// The gate names `--gate` computes alone, for the refusal message. -pub const NAMED_GATES: [&str; 4] = ["relations", "shapes", "sigma", "validate"]; +pub const NAMED_GATES: [&str; 7] = [ + "depends-on-present", + "relations", + "shapes", + "sigma", + "theorem-pairing", + "valid-under", + "validate", +]; + +/// The EV-11 ratchet gates, by name, in the order the full run computes them. +type RatchetRun = fn(&Path) -> ratchet_gates::RatchetOutcome; +const RATCHET_GATES: [(&str, RatchetRun); 2] = [ + ("theorem-pairing", ratchet_gates::run_theorem_pairing_gate), + ( + "depends-on-present", + ratchet_gates::run_depends_on_present_gate, + ), +]; + +/// An EV-11 ratchet as `run_lint` reports it. Its decline becomes a SKIPPED gate here, as sigma's non-verdict +/// answers do — under `--gate ` it is an exit of its own (2). +fn ratchet_result( + contract_dir: &Path, + validation_passed: bool, + name: &str, + run: RatchetRun, +) -> (GateResult, Vec) { + if !validation_passed { + return (skipped_gate(name, "validation failed"), Vec::new()); + } + match run(contract_dir) { + ratchet_gates::RatchetOutcome::Ran { result, findings } => (*result, findings), + ratchet_gates::RatchetOutcome::Declined(why) => (skipped_gate(name, &why), Vec::new()), + } +} /// The `sigma` gate as `run_lint` reports it. Σ's two non-verdict answers become SKIPPED gates here — under /// `--gate sigma` they are an exit of their own (decline / error), but inside a full run "skipped" is how the @@ -701,6 +877,35 @@ fn sigma_result(contract_dir: &Path, validation_passed: bool) -> (GateResult, Ve } } +/// The `valid-under` gate as `run_lint` reports it (ONT-7). Its three non-verdict answers become SKIPPED gates +/// here, as sigma's do — under `--gate valid-under` they are exits of their own (decline / error). +fn valid_under_result( + contract_dir: &Path, + validation_passed: bool, +) -> (GateResult, Vec) { + if !validation_passed { + return (skipped_gate("valid-under", "validation failed"), Vec::new()); + } + match valid_under_gate::run_valid_under_gate(contract_dir) { + valid_under_gate::ValidUnderOutcome::Ran { result, findings } => (*result, findings), + valid_under_gate::ValidUnderOutcome::NoSigma => ( + skipped_gate("valid-under", "no contracts/ontology.yaml"), + Vec::new(), + ), + valid_under_gate::ValidUnderOutcome::Malformed(e) => ( + skipped_gate("valid-under", &format!("Σ is malformed: {e}")), + Vec::new(), + ), + valid_under_gate::ValidUnderOutcome::NoKernels { contracts_checked } => ( + skipped_gate( + "valid-under", + &format!("no kernel-kind contract and no valid_under in {contracts_checked} contracts — R-2: zero is a decline"), + ), + Vec::new(), + ), + } +} + /// The `relations` gate as `run_lint` reports it. Its three non-verdict answers become SKIPPED gates here, as /// sigma's do — under `--gate relations` they are exits of their own (decline / error). fn relations_result( @@ -755,10 +960,18 @@ fn shapes_result(contract_dir: &Path, validation_passed: bool) -> (GateResult, V skipped_gate("shapes", &format!("extract:parity-receipt matched {found} focus node(s) and evidence/parity/EXPECTED_RECEIPTS says {expected} ({shapes_n} shape(s)){} — an extractor that saw the wrong corpus reports the same \"no violations\" as one that saw all of it", if refused.is_empty() { String::new() } else { format!("; refused: {}", refused.join("; ")) })), Vec::new(), ), + shapes_gate::ShapesOutcome::HarnessBroken { causes } => ( + skipped_gate("shapes", &format!("the CRUX harness measured itself, not apr: {}", causes.join("; "))), + Vec::new(), + ), shapes_gate::ShapesOutcome::NoReceipts { shapes_n, dir } => ( skipped_gate("shapes", &format!("{shapes_n} shape(s) resolve receipts and the tree holds none under {dir}/ — R-2: unmeasured is a decline")), Vec::new(), ), + shapes_gate::ShapesOutcome::EmptyDomain { shapes_n } => ( + skipped_gate("shapes", &format!("capability-cells: the required-cell domain D is empty ({shapes_n} shape(s)) — R-2: zero is a decline, never a pass")), + Vec::new(), + ), shapes_gate::ShapesOutcome::PositiveControlFailed { shapes_n, focus_nodes_n, which } => ( skipped_gate("shapes", &format!("positive control {which} did not fire ({shapes_n} shape(s), {focus_nodes_n} focus node(s)) — the gate cannot reject")), Vec::new(), diff --git a/crates/aprender-contracts/src/lint/mod_tests.rs b/crates/aprender-contracts/src/lint/mod_tests.rs index 398a43f707..0081304067 100644 --- a/crates/aprender-contracts/src/lint/mod_tests.rs +++ b/crates/aprender-contracts/src/lint/mod_tests.rs @@ -10,9 +10,10 @@ fn lint_passes_on_real_contracts() { let config = LintConfig::new(&dir, None, 0.0); let report = run_lint(&config); assert!(report.passed, "lint should pass: {report:?}"); - // 12 gates: validate, audit, score, verify, enforce, enforcement-level, reverse-coverage, - // duplicate-stems (PV-DUP-001), composition, sigma (ONT-2b), relations (ONT-4), shapes (ONT-4b). - assert_eq!(report.gates.len(), 12); + // 15 gates: validate, audit, score, verify, enforce, enforcement-level, reverse-coverage, + // duplicate-stems (PV-DUP-001), composition, sigma (ONT-2b), relations (ONT-4), shapes (ONT-4b), + // valid-under (ONT-7), theorem-pairing and depends-on-present (PVL-001 EV-11). + assert_eq!(report.gates.len(), 15); } #[test] @@ -26,10 +27,72 @@ fn lint_score_gate_fails_with_high_threshold() { #[test] fn lint_empty_dir() { + // The lint takes the contract dir's PARENT as the project root and reads + // `scripts/contract_duplicate_stem_baseline.txt` from it. A bare tempdir's parent is + // the shared `/tmp`, so a stray `/tmp/scripts/` failed this test (#4207). Nest it. let tmp = tempfile::tempdir().unwrap(); - let config = LintConfig::new(tmp.path(), None, 0.0); + let dir = tmp.path().join("contracts"); + std::fs::create_dir_all(&dir).unwrap(); + let config = LintConfig::new(&dir, None, 0.0); let report = run_lint(&config); - assert!(report.passed); + assert!(report.passed, "empty dir should pass: {report:?}"); + // The new-finding state is the fixture's own, not `$TMPDIR/.pv` shared with every other run (#4173). + assert_eq!(pv_state_dir(&dir), tmp.path().join(".pv")); + assert!(tmp.path().join(".pv/lint-previous.json").is_file()); +} + +/// #4173: `pv_state_dir` is the contract dir's PARENT, so a test that lints a bare `tempdir()` reads +/// and writes `$TMPDIR/.pv/lint-previous.json`, a file every other run on the host shares. Its +/// verdict then depends on state it does not own. Every lint test must nest its corpus. +#[test] +fn no_lint_test_points_the_lint_at_a_bare_tempdir() { + let bare = |line: &str| { + let l: String = line.chars().filter(|c| !c.is_whitespace()).collect(); + ["(tmp.path()", "(&tmp.path()"].iter().any(|arg| { + let pat = format!("{}{arg}", concat!("LintConfig::", "new")); + l.match_indices(&pat) + .any(|(k, _)| matches!(l[k + pat.len()..].chars().next(), Some(',' | ')'))) + }) + }; + for (line, want) in [ + ( + concat!("run_lint(&LintConfig::new", "(tmp.path(), None, 0.0));"), + true, + ), + ( + concat!("let c = LintConfig::new", "( &tmp.path(), None, 0.0);"), + true, + ), + ("let c = LintConfig::new(&corpus, None, 0.0);", false), + ( + "let c = LintConfig::new(&tmp.path().join(\"contracts\"), None, 0.0);", + false, + ), + ] { + assert_eq!(bare(line), want, "case table: {line}"); + } + let lint_src = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/lint"); + let mut scanned = 0; + for entry in std::fs::read_dir(&lint_src).unwrap() { + let path = entry.unwrap().path(); + if path.extension().is_some_and(|e| e == "rs") { + scanned += 1; + let src = std::fs::read_to_string(&path).unwrap(); + for (i, line) in src.lines().enumerate() { + assert!( + !bare(line), + "{}:{}: lints a bare tempdir: {line}", + path.display(), + i + 1 + ); + } + } + } + assert!( + scanned > 5, + "scanned only {scanned} files under {}", + lint_src.display() + ); } #[test] @@ -155,14 +218,16 @@ fn lint_cache_second_run_hits() { #[test] fn lint_validation_failure_skips_audit_and_score() { let tmp = tempfile::tempdir().unwrap(); + let dir = tmp.path().join("contracts"); + std::fs::create_dir_all(&dir).unwrap(); // Write a malformed YAML that will parse into a Contract with validation errors // Actually: write something that fails to parse entirely - std::fs::write(tmp.path().join("bad.yaml"), "not: valid: yaml: {{{{").unwrap(); - let config = LintConfig::new(tmp.path(), None, 0.0); + std::fs::write(dir.join("bad.yaml"), "not: valid: yaml: {{{{").unwrap(); + let config = LintConfig::new(&dir, None, 0.0); let report = run_lint(&config); assert!(!report.passed); // validate should fail, all subsequent gates should be skipped - assert_eq!(report.gates.len(), 12); + assert_eq!(report.gates.len(), 15); assert!(!report.gates[0].passed); // validate failed assert!(report.gates[1].skipped); // audit skipped assert!(report.gates[2].skipped); // score skipped @@ -368,7 +433,12 @@ fn every_gate_verdict_agrees_with_passed_and_skipped_on_the_real_corpus() { "reverse-coverage".to_string(), "sigma".to_string(), "relations".to_string(), - "shapes".to_string() + "shapes".to_string(), + // ONT-7, R-8: computed in every run, armed only when the baseline names it. + "valid-under".to_string(), + // PVL-001 EV-11: computed in every run (R-8), armed per repo. + "theorem-pairing".to_string(), + "depends-on-present".to_string(), ] ); } @@ -403,3 +473,45 @@ fn a_gate_a_flag_ran_is_reported_but_not_armed() { .any(|g| g.name == "strict-test-binding")); assert_eq!(report.armed_gates.len(), 8); } + +/// ONT-7, R-8: gate 13 is COMPUTED when validation passes and SKIPPED, naming why, when it fails. Both halves +/// at the lib level, because the CI mutation lane runs `--lib` only (#4076 round-2 review): a mutant that +/// inverts `validation_passed` in `valid_under_result` must fail here, not only in the CLI integration test. +#[test] +fn valid_under_is_computed_when_validation_passes_and_skipped_when_it_fails() { + let report = run_lint(&LintConfig::new(&contracts_dir(), None, 0.0)); + let g = report + .gates + .iter() + .find(|g| g.name == "valid-under") + .expect("gate 13 is in every run"); + assert!( + !g.skipped && g.passed, + "computed and Pass on the repo corpus: {g:?}" + ); + + // Σ and a kernel contract are present, so the ONLY reason to skip is the failed validation. + // Nested, not the bare tempdir: the lint writes its state into the contract dir's PARENT (#4173). + let tmp = tempfile::tempdir().unwrap(); + let corpus = tmp.path().join("contracts"); + std::fs::create_dir_all(&corpus).unwrap(); + let fixture = contracts_dir().join("../tests/fixtures/ont/valid-under-ok"); + for f in ["ontology.yaml", "fixture-vu-v1.yaml"] { + std::fs::copy(fixture.join(f), corpus.join(f)).unwrap(); + } + std::fs::write(corpus.join("bad.yaml"), "not: valid: yaml: {{{{").unwrap(); + let report = run_lint(&LintConfig::new(&corpus, None, 0.0)); + let g = report + .gates + .iter() + .find(|g| g.name == "valid-under") + .expect("gate 13 is in every run"); + assert!( + g.skipped, + "validation failed, so the gate is skipped: {g:?}" + ); + assert!( + matches!(&g.detail, GateDetail::Skipped { reason } if reason == "validation failed"), + "{g:?}" + ); +} diff --git a/crates/aprender-contracts/src/lint/ratchet_gates.rs b/crates/aprender-contracts/src/lint/ratchet_gates.rs new file mode 100644 index 0000000000..fb984f12ea --- /dev/null +++ b/crates/aprender-contracts/src/lint/ratchet_gates.rs @@ -0,0 +1,314 @@ +//! PVL-001 §EV-11 — two shrink-only ratchets over `contracts/lint-baseline.json`. +//! +//! - `theorem-pairing` (PV-RAT-001): a Lean THEOREM MODULE is a `.lean` file under +//! `/ProvableContracts/Theorems/`, named by its dotted path from the base +//! (`ProvableContracts.Theorems.Softmax.PartitionOfUnity`). It is PAIRED when that full name appears, on +//! identifier boundaries, in a `.md` file under `book/` or `crates/aprender-contracts-staging/book/` of the repo +//! root (the contract dir's parent). The debt `unpaired_theorem_modules` may not rise. +//! - `depends-on-present` (PV-RAT-002): a kernel-kind contract (the effective `kind()`, which reads registries as Registry — the class +//! the valid-under gate obliges) with an empty `metadata.depends_on`. The debt `contracts_without_depends_on` +//! may not rise. +//! +//! THE PAIRING RULE IS STRICT ON PURPOSE. The spec row says "module name appears in `book/`"; the full dotted +//! name is the only reading that cannot be satisfied by accident — the book mentions the file STEM of 80 of the +//! 165 `.lean` files (measured 2026-09-24), mostly as ordinary words, and the full module name of 1 of 131. The root module and the lakefile are not +//! theorem modules, so they are not counted either way. +//! +//! THE GATES NEVER WRITE. At or below the baseline they PASS and report the count; lowering the recorded number +//! is `make lint-ratchet`'s job (never in CI). No baseline key is `Unknown(Report)`, never a pass: the whole +//! verdict is the comparison, so the count is REPORTED (that is how `make lint-ratchet` records the first +//! baseline) and not judged, and `--gate` exits 2. A Lean base with no theorem module, a repo with no book, or a +//! corpus with no kernel-kind contract measured nothing and declines outright (ONT R-2: zero is a decline). + +use std::path::{Path, PathBuf}; +use std::time::Instant; + +use crate::ontology::extract::lean::base_under; +use crate::schema::{parse_contract, ContractKind}; + +use super::finding::LintFinding; +use super::rules::RuleSeverity; +use super::{GateDetail, GateExtra, GateResult, Verdict}; +use crate::ontology::verdict::Reason; + +/// Top-level `lint-baseline.json` key for the theorem-pairing debt. +pub const UNPAIRED_KEY: &str = "unpaired_theorem_modules"; +/// Top-level `lint-baseline.json` key for the depends_on debt. +pub const WITHOUT_DEPENDS_ON_KEY: &str = "contracts_without_depends_on"; +/// The book roots a theorem module is paired against, relative to the repo root. +pub const BOOK_ROOTS: [&str; 2] = ["book", "crates/aprender-contracts-staging/book"]; + +/// What one ratchet gate run answers. Only [`RatchetOutcome::Ran`] is a verdict about the corpus. +#[derive(Debug)] +pub enum RatchetOutcome { + /// Nothing was measured, or nothing can be compared: the reason, for stderr (ONT R-2: zero is a decline). + Declined(String), + /// The gate measured and compared. + Ran { + result: Box, + findings: Vec, + }, +} + +/// The top-level integer `key` of `/lint-baseline.json`. Absent, unreadable, or not a +/// non-negative integer → `None`. +fn baseline_of(contract_dir: &Path, key: &str) -> Option { + let raw = std::fs::read_to_string(contract_dir.join("lint-baseline.json")).ok()?; + let doc: serde_json::Value = serde_json::from_str(&raw).ok()?; + doc.get(key)?.as_u64().and_then(|n| usize::try_from(n).ok()) +} + +/// The repo root the contract dir sits in: its parent, or `.` for a bare relative name. +fn repo_root(contract_dir: &Path) -> PathBuf { + match contract_dir.parent() { + Some(p) if !p.as_os_str().is_empty() => p.to_path_buf(), + _ => PathBuf::from("."), + } +} + +/// Every file under `dir` with extension `ext`, recursively, in byte order. Unreadable directories are skipped. +fn files_with_ext(dir: &Path, ext: &str, out: &mut Vec) { + let Ok(entries) = std::fs::read_dir(dir) else { + return; + }; + for entry in entries.flatten() { + let path = entry.path(); + if path.is_dir() { + files_with_ext(&path, ext, out); + } else if path.extension().and_then(|e| e.to_str()) == Some(ext) { + out.push(path); + } + } +} + +/// `/ProvableContracts/Theorems/A/B.lean` → `ProvableContracts.Theorems.A.B`. +fn module_name(base: &Path, file: &Path) -> Option { + let rel = file.strip_prefix(base).ok()?.with_extension(""); + let parts: Vec<&str> = rel + .components() + .filter_map(|c| c.as_os_str().to_str()) + .collect(); + Some(parts.join(".")) +} + +fn is_ident_char(c: char) -> bool { + c.is_alphanumeric() || c == '_' || c == '\'' +} + +/// Does `text` contain `name` with identifier boundaries on both sides? A trailing `.` counts as a boundary +/// only when no identifier follows it, so `…Softmax.Partition` does not pair `…Softmax.PartitionOfUnity` or a +/// deeper module, and a sentence ending in the module name still pairs it. +#[must_use] +pub fn mentions_module(text: &str, name: &str) -> bool { + let mut from = 0; + while let Some(off) = text[from..].find(name) { + let start = from + off; + let end = start + name.len(); + let before_ok = text[..start] + .chars() + .next_back() + .is_none_or(|c| !is_ident_char(c) && c != '.'); + let mut after = text[end..].chars(); + let after_ok = match after.next() { + None => true, + Some('.') => after.next().is_none_or(|c| !is_ident_char(c)), + Some(c) => !is_ident_char(c), + }; + if before_ok && after_ok { + return true; + } + from = start + name.chars().next().map_or(1, char::len_utf8); + } + false +} + +/// PV-RAT-001 / PV-RAT-002: the debt may fall, never rise. No baseline → no finding (the verdict says why). +fn ratchet_finding( + rule: &str, + key: &str, + baseline: Option, + now: usize, + fix: &str, +) -> Option { + let baseline = baseline?; + if now <= baseline { + return None; + } + let mut f = LintFinding::new( + rule, + RuleSeverity::Error, + format!( + "{key} rose {baseline} -> {now}: {fix}. The baseline in contracts/lint-baseline.json is shrink-only; `make lint-ratchet` only lowers it" + ), + "contracts/lint-baseline.json".to_string(), + ); + f.contract_stem = Some("lint-baseline".to_string()); + Some(f) +} + +fn result_of( + name: &str, + checked: usize, + findings: &[LintFinding], + baseline: Option, + start: Instant, + extra: GateExtra, +) -> Box { + let violations = findings.len(); + let judged = baseline.is_some(); + let passed = judged && violations == 0; + Box::new(GateResult { + name: name.into(), + passed, + skipped: !judged, + verdict: if judged { + Verdict::from_gate(passed, false) + } else { + Verdict::Unknown(Reason::Report) + }, + duration_ms: u64::try_from(start.elapsed().as_millis()).unwrap_or(u64::MAX), + // `GateDetail` is FROZEN at the 0.3.1 variants (see `GateExtra`); the payload rides in `GateExtra`. + detail: GateDetail::Validate { + contracts: checked, + errors: violations, + warnings: 0, + error_messages: findings.iter().map(|f| f.message.clone()).collect(), + }, + extra: Some(extra), + }) +} + +/// The `theorem-pairing` gate over `contract_dir`'s repo root. +#[must_use] +pub fn run_theorem_pairing_gate(contract_dir: &Path) -> RatchetOutcome { + let start = Instant::now(); + let root = repo_root(contract_dir); + let Some(base) = base_under(&root) else { + return RatchetOutcome::Declined(format!( + "no Lean theorem base (ProvableContracts/Theorems) under {}", + root.display() + )); + }; + let mut lean = Vec::new(); + files_with_ext(&base.join("ProvableContracts/Theorems"), "lean", &mut lean); + lean.sort(); + let modules: Vec = lean.iter().filter_map(|f| module_name(&base, f)).collect(); + if modules.is_empty() { + return RatchetOutcome::Declined(format!("no theorem module under {}", base.display())); + } + let books: Vec = BOOK_ROOTS + .iter() + .map(|b| root.join(b)) + .filter(|p| p.is_dir()) + .collect(); + let mut pages = Vec::new(); + for b in &books { + files_with_ext(b, "md", &mut pages); + } + if pages.is_empty() { + return RatchetOutcome::Declined(format!( + "no book page (*.md) under {} of {}: nothing to pair {} theorem module(s) against", + BOOK_ROOTS.join(" or "), + root.display(), + modules.len() + )); + } + let baseline = baseline_of(contract_dir, UNPAIRED_KEY); + let texts: Vec = pages + .iter() + .filter_map(|p| std::fs::read_to_string(p).ok()) + .collect(); + let unpaired: Vec = modules + .iter() + .filter(|m| !texts.iter().any(|t| mentions_module(t, m))) + .cloned() + .collect(); + let findings: Vec = ratchet_finding( + "PV-RAT-001", + UNPAIRED_KEY, + baseline, + unpaired.len(), + "a Lean theorem module was added that no book page names — cite its full module name in book/", + ) + .into_iter() + .collect(); + let extra = GateExtra::TheoremPairing { + lean_base: base.display().to_string(), + book_pages: pages.len(), + theorem_modules: modules.len(), + paired: modules.len() - unpaired.len(), + unpaired_theorem_modules: unpaired.len(), + baseline, + unpaired, + violations: findings.len(), + }; + RatchetOutcome::Ran { + result: result_of( + "theorem-pairing", + modules.len(), + &findings, + baseline, + start, + extra, + ), + findings, + } +} + +/// The `depends-on-present` gate over `contract_dir`. +#[must_use] +pub fn run_depends_on_present_gate(contract_dir: &Path) -> RatchetOutcome { + let start = Instant::now(); + let mut files = Vec::new(); + super::collect_yaml_files(contract_dir, &mut files); + files.sort(); + let (mut checked, mut kernels, mut without) = (0usize, 0usize, 0usize); + for file in &files { + let Ok(contract) = parse_contract(file) else { + continue; + }; + checked += 1; + // `kind()` already reads a `registry: true` kernel as Registry, so Kernel here is never a registry. + if contract.kind() == ContractKind::Kernel { + kernels += 1; + without += usize::from(contract.metadata.depends_on.is_empty()); + } + } + if kernels == 0 { + return RatchetOutcome::Declined(format!( + "no kernel-kind contract in {checked} contract(s): nothing was measured" + )); + } + let baseline = baseline_of(contract_dir, WITHOUT_DEPENDS_ON_KEY); + let findings: Vec = ratchet_finding( + "PV-RAT-002", + WITHOUT_DEPENDS_ON_KEY, + baseline, + without, + "a kernel-kind contract with an empty `metadata.depends_on` was added — name what it composes", + ) + .into_iter() + .collect(); + let extra = GateExtra::DependsOnPresent { + contracts_checked: checked, + kernel_contracts: kernels, + contracts_without_depends_on: without, + baseline, + violations: findings.len(), + }; + RatchetOutcome::Ran { + result: result_of( + "depends-on-present", + checked, + &findings, + baseline, + start, + extra, + ), + findings, + } +} + +#[cfg(test)] +#[path = "ratchet_gates_tests.rs"] +mod tests; diff --git a/crates/aprender-contracts/src/lint/ratchet_gates_tests.rs b/crates/aprender-contracts/src/lint/ratchet_gates_tests.rs new file mode 100644 index 0000000000..4aad764e32 --- /dev/null +++ b/crates/aprender-contracts/src/lint/ratchet_gates_tests.rs @@ -0,0 +1,333 @@ +//! PVL-001 EV-11 — the two `pv lint` ratchets, each case on a throwaway repo built in a tempdir. + +use super::*; + +const THEOREMS: &str = "crates/aprender-contracts-staging/lean/ProvableContracts/Theorems"; + +fn write(root: &Path, rel: &str, text: &str) { + let p = root.join(rel); + std::fs::create_dir_all(p.parent().expect("has a parent")).expect("mkdir"); + std::fs::write(p, text).expect("write"); +} + +fn baseline(root: &Path, json: &str) { + write(root, "contracts/lint-baseline.json", json); +} + +/// A repo with two theorem modules, one of them named by a book page. +fn pairing_repo() -> tempfile::TempDir { + let t = tempfile::tempdir().expect("tempdir"); + write( + t.path(), + &format!("{THEOREMS}/Softmax/PartitionOfUnity.lean"), + "theorem p : True := trivial\n", + ); + write( + t.path(), + &format!("{THEOREMS}/Softmax/Bounds.lean"), + "theorem b : True := trivial\n", + ); + write( + t.path(), + "book/src/softmax.md", + "The partition law is `ProvableContracts.Theorems.Softmax.PartitionOfUnity`.\n", + ); + t +} + +fn pairing(root: &Path) -> (GateResult, Vec) { + match run_theorem_pairing_gate(&root.join("contracts")) { + RatchetOutcome::Ran { result, findings } => { + (*result, findings.into_iter().map(|f| f.rule_id).collect()) + } + RatchetOutcome::Declined(why) => panic!("expected a verdict, got a decline: {why}"), + } +} + +fn unpaired_of(r: &GateResult) -> (usize, Vec) { + match r.extra.as_ref() { + Some(GateExtra::TheoremPairing { + unpaired_theorem_modules, + unpaired, + .. + }) => (*unpaired_theorem_modules, unpaired.clone()), + other => panic!("expected TheoremPairing, got {other:?}"), + } +} + +// ── mentions_module: identifier boundaries ──────────────────────────────────────────────────────────────── + +#[test] +fn a_module_is_mentioned_only_on_identifier_boundaries() { + let m = "P.T.Softmax.Partition"; + for (text, want) in [ + ("see P.T.Softmax.Partition here", true), + ("`P.T.Softmax.Partition`", true), + ("ends the sentence P.T.Softmax.Partition.", true), + ("P.T.Softmax.Partition", true), + ("(P.T.Softmax.Partition)", true), + // a LONGER module is not this one + ("P.T.Softmax.PartitionOfUnity", false), + ("P.T.Softmax.Partition.Deeper", false), + ("P.T.Softmax.Partition_2", false), + ("P.T.Softmax.Partition'", false), + // a PREFIXED name is not this one + ("X.P.T.Softmax.Partition", false), + ("XP.T.Softmax.Partition", false), + // the file stem alone is not the module + ("Partition", false), + ("", false), + ] { + assert_eq!(mentions_module(text, m), want, "{text:?}"); + } + // a bad first occurrence does not hide a good later one + assert!(mentions_module( + "P.T.Softmax.PartitionX and P.T.Softmax.Partition", + m + )); +} + +#[test] +fn a_module_name_is_the_dotted_path_from_the_base() { + let base = Path::new("/r/lean"); + assert_eq!( + module_name( + base, + Path::new("/r/lean/ProvableContracts/Theorems/A/B.lean") + ) + .as_deref(), + Some("ProvableContracts.Theorems.A.B") + ); + assert_eq!(module_name(base, Path::new("/elsewhere/A.lean")), None); +} + +// ── the ratchet itself ──────────────────────────────────────────────────────────────────────────────────── + +#[test] +fn the_ratchet_rejects_a_rise_only() { + let f = |b, n| ratchet_finding("PV-RAT-001", UNPAIRED_KEY, b, n, "fix").map(|f| f.rule_id); + assert_eq!(f(Some(5), 4), None, "a fall passes"); + assert_eq!(f(Some(5), 5), None, "equal passes"); + assert_eq!(f(Some(5), 6).as_deref(), Some("PV-RAT-001")); + assert_eq!(f(None, 1000), None, "no baseline: reported, not judged"); +} + +#[test] +fn a_baseline_that_is_not_a_non_negative_integer_is_no_baseline() { + let t = tempfile::tempdir().expect("tempdir"); + let dir = t.path().join("contracts"); + for (json, want) in [ + (r#"{"unpaired_theorem_modules": 7}"#, Some(7)), + (r#"{"unpaired_theorem_modules": -1}"#, None), + (r#"{"unpaired_theorem_modules": "7"}"#, None), + (r#"{"ont": {"unpaired_theorem_modules": 7}}"#, None), + ("not json", None), + ] { + baseline(t.path(), json); + assert_eq!(baseline_of(&dir, UNPAIRED_KEY), want, "{json}"); + } +} + +// ── theorem-pairing over a repo ─────────────────────────────────────────────────────────────────────────── + +#[test] +fn theorem_pairing_counts_the_modules_no_book_page_names() { + let t = pairing_repo(); + baseline(t.path(), r#"{"unpaired_theorem_modules": 1}"#); + let (r, rules) = pairing(t.path()); + assert!(r.passed && !r.skipped, "at the baseline: pass"); + assert!(rules.is_empty()); + assert_eq!( + unpaired_of(&r), + ( + 1, + vec!["ProvableContracts.Theorems.Softmax.Bounds".to_string()] + ) + ); +} + +/// The spec row's mutation, verbatim: "add an unpaired Theorem module → RED". +#[test] +fn adding_an_unpaired_theorem_module_is_red() { + let t = pairing_repo(); + baseline(t.path(), r#"{"unpaired_theorem_modules": 1}"#); + write( + t.path(), + &format!("{THEOREMS}/Gelu/Tanh.lean"), + "theorem g : True := trivial\n", + ); + let (r, rules) = pairing(t.path()); + assert!(!r.passed); + assert_eq!(r.verdict, Verdict::from_gate(false, false)); + assert_eq!(rules, ["PV-RAT-001"]); + assert_eq!(unpaired_of(&r).0, 2); +} + +#[test] +fn naming_the_module_in_the_staging_book_pairs_it_too() { + let t = pairing_repo(); + baseline(t.path(), r#"{"unpaired_theorem_modules": 0}"#); + write( + t.path(), + "crates/aprender-contracts-staging/book/src/bounds.md", + "ProvableContracts.Theorems.Softmax.Bounds\n", + ); + let (r, rules) = pairing(t.path()); + assert!(r.passed, "{rules:?}"); + assert_eq!(unpaired_of(&r).0, 0); +} + +#[test] +fn the_file_stem_or_a_longer_name_does_not_pair_a_module() { + let t = pairing_repo(); + baseline(t.path(), r#"{"unpaired_theorem_modules": 1}"#); + write( + t.path(), + "book/src/more.md", + "Bounds, Softmax.Bounds and ProvableContracts.Theorems.Softmax.BoundsTight are all different.\n", + ); + assert_eq!(unpaired_of(&pairing(t.path()).0).0, 1); +} + +#[test] +fn no_baseline_is_reported_and_never_a_pass() { + let t = pairing_repo(); + let (r, rules) = pairing(t.path()); + assert!(!r.passed && r.skipped); + assert_eq!(r.verdict, Verdict::Unknown(Reason::Report)); + assert!( + rules.is_empty(), + "nothing to compare against, so no finding" + ); + assert_eq!(unpaired_of(&r).0, 1, "the count is still reported"); +} + +#[test] +fn theorem_pairing_declines_when_nothing_was_measured() { + // no Lean base at all + let t = tempfile::tempdir().expect("tempdir"); + write(t.path(), "book/a.md", "x\n"); + assert!(matches!( + run_theorem_pairing_gate(&t.path().join("contracts")), + RatchetOutcome::Declined(_) + )); + // a base with no theorem module + std::fs::create_dir_all(t.path().join(THEOREMS)).expect("mkdir"); + assert!(matches!( + run_theorem_pairing_gate(&t.path().join("contracts")), + RatchetOutcome::Declined(_) + )); + // theorem modules but no book page + let t = pairing_repo(); + std::fs::remove_dir_all(t.path().join("book")).expect("rm book"); + match run_theorem_pairing_gate(&t.path().join("contracts")) { + RatchetOutcome::Declined(why) => assert!(why.contains("no book page"), "{why}"), + RatchetOutcome::Ran { .. } => panic!("no book is a decline, not 2 unpaired"), + } +} + +// ── depends-on-present over a corpus ────────────────────────────────────────────────────────────────────── + +fn contract(kind: &str, depends_on: &str) -> String { + format!( + "metadata:\n version: \"1.0.0\"\n kind: {kind}\n description: EV-11 fixture\n depends_on: {depends_on}\n\ + equations:\n identity:\n formula: \"len(out) = len(x)\"\n" + ) +} + +fn depends(root: &Path) -> (GateResult, Vec) { + match run_depends_on_present_gate(&root.join("contracts")) { + RatchetOutcome::Ran { result, findings } => { + (*result, findings.into_iter().map(|f| f.rule_id).collect()) + } + RatchetOutcome::Declined(why) => panic!("expected a verdict, got a decline: {why}"), + } +} + +fn without_of(r: &GateResult) -> (usize, usize) { + match r.extra.as_ref() { + Some(GateExtra::DependsOnPresent { + kernel_contracts, + contracts_without_depends_on, + .. + }) => (*kernel_contracts, *contracts_without_depends_on), + other => panic!("expected DependsOnPresent, got {other:?}"), + } +} + +fn depends_repo() -> tempfile::TempDir { + let t = tempfile::tempdir().expect("tempdir"); + write( + t.path(), + "contracts/a-v1.yaml", + &contract("kernel", "[b-v1]"), + ); + write(t.path(), "contracts/b-v1.yaml", &contract("kernel", "[]")); + // neither of these is a kernel contract, so neither counts either way + write( + t.path(), + "contracts/reg-v1.yaml", + &contract("registry", "[]"), + ); + write( + t.path(), + "contracts/pat-v1.yaml", + &contract("pattern", "[]"), + ); + t +} + +#[test] +fn depends_on_counts_kernel_contracts_with_none() { + let t = depends_repo(); + baseline(t.path(), r#"{"contracts_without_depends_on": 1}"#); + let (r, rules) = depends(t.path()); + assert!(r.passed && !r.skipped, "{rules:?}"); + assert_eq!( + without_of(&r), + (2, 1), + "registry and pattern are not kernel contracts" + ); +} + +#[test] +fn adding_a_kernel_contract_with_no_depends_on_is_red() { + let t = depends_repo(); + baseline(t.path(), r#"{"contracts_without_depends_on": 1}"#); + write(t.path(), "contracts/c-v1.yaml", &contract("kernel", "[]")); + let (r, rules) = depends(t.path()); + assert!(!r.passed); + assert_eq!(rules, ["PV-RAT-002"]); + assert_eq!(without_of(&r), (3, 2)); +} + +#[test] +fn depends_on_declines_with_no_kernel_contract() { + let t = tempfile::tempdir().expect("tempdir"); + write( + t.path(), + "contracts/pat-v1.yaml", + &contract("pattern", "[]"), + ); + assert!(matches!( + run_depends_on_present_gate(&t.path().join("contracts")), + RatchetOutcome::Declined(_) + )); +} + +// ── the gates read the baseline and never write it ─────────────────────────────────────────────────────── + +#[test] +fn neither_gate_writes_the_baseline() { + let t = pairing_repo(); + write(t.path(), "contracts/b-v1.yaml", &contract("kernel", "[]")); + let json = "{\n \"unpaired_theorem_modules\": 9,\n \"contracts_without_depends_on\": 9\n}\n"; + baseline(t.path(), json); + // both counts FELL (1 < 9): a gate that "helpfully" lowered the baseline would change the file + assert!(pairing(t.path()).0.passed); + assert!(depends(t.path()).0.passed); + assert_eq!( + std::fs::read_to_string(t.path().join("contracts/lint-baseline.json")).expect("read"), + json + ); +} diff --git a/crates/aprender-contracts/src/lint/shapes_gate.rs b/crates/aprender-contracts/src/lint/shapes_gate.rs index ac6c6242a9..12e8e5c774 100644 --- a/crates/aprender-contracts/src/lint/shapes_gate.rs +++ b/crates/aprender-contracts/src/lint/shapes_gate.rs @@ -33,10 +33,11 @@ use std::path::Path; use std::time::Instant; use crate::ontology::arming::ArmedShapes; +use crate::ontology::capability_cells; use crate::ontology::extract::release_inputs::Subject; use crate::ontology::extract::{ - self, apr_model, code, gguf, json, lean, parity_receipt, pv_contract, release_evidence, - ExtractFailure, + self, apr_model, cli_surface, code, gguf, json, lean, parity_receipt, pv_contract, + release_evidence, ExtractFailure, }; use crate::ontology::rdf::{iri, Graph, Term, RDF_TYPE}; use crate::ontology::receipts; @@ -44,6 +45,7 @@ use crate::ontology::shapes::{self, NodeShape, Report, Severity, ShapeError}; use crate::ontology::verdict::Reason; use crate::ontology::w3c; +use super::capability_cells_gate as cells_gate; use super::finding::LintFinding; use super::rules::RuleSeverity; use super::{GateDetail, GateExtra, GateResult, Verdict}; @@ -76,6 +78,9 @@ pub enum ShapesOutcome { found: usize, refused: Vec, }, + /// ONT-4c5: a contract declares `capability-cells` and its required-cell domain D is empty — R-2's decline + /// (exit 2 with a `decline:` line), never Pass and never RED. + EmptyDomain { shapes_n: usize }, /// A positive control did not fire. PositiveControlFailed { shapes_n: usize, @@ -91,6 +96,9 @@ pub enum ShapesOutcome { n: usize, failed: Vec, }, + /// #3739: the CRUX harness measured itself, not apr — a model's positive-control prompt came back ALL_WRONG, + /// or a model owing CRUX cells has no measured control. Declined (exit 2), each model and cause named. + HarnessBroken { causes: Vec }, /// Shapes ran over the corpus, and the controls fired. Ran { result: Box, @@ -249,28 +257,44 @@ pub fn run_shapes_gate_with(contract_dir: &Path, opts: &ShapesOptions) -> Shapes Err(answer) => return answer, }; - // ONE walk (R-18): every extractor, the json documents, the ladder receipts joined to the rungs — the same - // graph `pv extract` writes — plus, when a release subject is given, the release evidence (#3715). - let extraction = match extract::all_with(contract_dir, opts.release.as_ref()) { + let extraction = match extract_gradeable(contract_dir, opts, shapes.len()) { Ok(x) => x, - Err(e) => return ShapesOutcome::ExtractFailed(e), + Err(answer) => return answer, }; - // PMAT-3577: the count is pinned before anything is graded. A miss here is not a corpus verdict. - if let Some(refusal) = parity_refusal(&extraction.parity, shapes.len()) { - return refusal; - } - let graph = &extraction.graph; + // ONT-4c5: the validator's half runs on the gate's own copy — `pv extract` keeps writing what was found + let mut owned = extraction.graph.clone(); + let cells = match cells_corpus(&mut owned, &shapes, &extraction) { + Ok(c) => c, + Err(answer) => return answer, + }; + let graph = &owned; + // #3610: the per-shape reach, computed BEFORE any verdict — did this shape grade anything at all? + let focus_of = focus_of(graph, &shapes); + let (vacuous_any, armed_vacuity) = vacuities(&focus_of, &shapes, &arming); let (mut report, plant_violations) = validate_with_plant(graph, &shapes, &arming); if opts.only.is_some() { order_by_family(&mut report, &shapes); } carry_extract_warnings(&mut report, &extraction.warnings); let pc_extract = extract_controls(); + let pc_shapes = match cells_controls( + &shapes, + &arming, + &report, + &extraction.gguf.rungs, + cells.as_ref(), + ) { + Ok(pc) => pc, + Err(answer) => return answer, + }; let unmeasured = needs_receipts(&shapes) && extraction.receipts.is_empty(); + // Every shape in scope empty is the global vacuity — unless every one of them declared it (#3610 quorum). + let all_allow_empty = shapes.iter().all(|s| s.allow_empty.is_some()); if let Some(d) = decline( shapes.len(), &report, + all_allow_empty, unmeasured, plant_violations, &pc_extract, @@ -282,21 +306,36 @@ pub fn run_shapes_gate_with(contract_dir: &Path, opts: &ShapesOptions) -> Shapes Err(differential) => return differential, }; - let counted = findings_of( + let mut counted = findings_of( &report, &arming, graph, &extraction.gguf, &extraction.apr_model, ); + // PV-ONT-013 NAMES the NotRun cells; it deliberately does not add to `counted.violations`. The verdict comes + // from the armed shape's own `maxCount 0` violation on the same edges, and `cells_controls` declines when the + // two disagree (wiring_holds), so the named list and the verdict cannot drift apart silently. + counted.findings.extend( + cells + .iter() + .flat_map(|cc| cells_gate::findings(cc, &arming)), + ); let passed = counted.violations == 0; - let verdict = verdict_of(&counted); - let by_shape = by_shape(graph, &shapes); + let verdict = verdict_of(&counted, armed_vacuity); + let by_shape = by_shape(&focus_of); + // A shape that graded nothing appears in NEITHER list: `armed_shapes` is the tool's claim about + // what it MEASURED, and `not_armed_shapes` means "not armed by policy". Filing a vacuity as a + // policy choice is how this defect hid, so it is named in `declines` only. let (armed_names, not_armed): (Vec, Vec) = shapes .iter() .map(|s| s.id.clone()) + .filter(|id| !vacuous_any.contains(id)) .partition(|id| arming.is_armed(id)); - let by_entity_type = by_entity_type(&extraction); + let by_entity_type = match by_entity_type(&extraction, &sigma_implemented(contract_dir)) { + Ok(m) => m, + Err(e) => return ShapesOutcome::Unsupported(e), + }; let duration = u64::try_from(start.elapsed().as_millis()).unwrap_or(u64::MAX); let result = GateResult { name: "shapes".into(), @@ -326,9 +365,20 @@ pub fn run_shapes_gate_with(contract_dir: &Path, opts: &ShapesOptions) -> Shapes triples: graph.len(), armed_shapes: armed_names, not_armed_shapes: not_armed, + // Every shape that graded NOTHING, armed or not — this is the ONLY list a + // vacuity appears in. An armed vacuity also drove the verdict to + // `Unknown(NoFocus)` above (nothing returns early there; the verdict is set + // and the run continues, so both armed and unarmed vacuities reach here). + // A field that can only ever be empty is decoration, which is the defect one + // layer up from this one. + declines: vacuous_any.clone(), unarmed_violations: counted.unarmed_violations, by_entity_type, - pc_extract, + controls: Box::new(super::ShapesControls { + pc_extract, + pc_shapes, + capability_cells: cells.as_ref().map(cells_gate::CapabilityCellsReport::from), + }), receipts: extraction.resolve.receipts, witnesses: extraction.resolve.witnesses, hex_mismatches: extraction.resolve.hex_mismatches, @@ -348,6 +398,83 @@ pub fn run_shapes_gate_with(contract_dir: &Path, opts: &ShapesOptions) -> Shapes } } +/// The ONE walk (R-18) and the refusals that must answer before anything is graded: every extractor, the json +/// documents, the ladder receipts joined to the rungs — the same graph `pv extract` writes — plus, when a release +/// subject is given, the release evidence (#3715). +fn extract_gradeable( + contract_dir: &Path, + opts: &ShapesOptions, + shapes_n: usize, +) -> Result { + let extraction = extract::all_with(contract_dir, opts.release.as_ref()) + .map_err(ShapesOutcome::ExtractFailed)?; + // PMAT-3577: the count is pinned before anything is graded. A miss here is not a corpus verdict. + if let Some(refusal) = parity_refusal(&extraction.parity, shapes_n) { + return Err(refusal); + } + // #3739 / cop: a broken CRUX harness is not a verdict about the release — decline, naming the model + match harness_broken(&extraction) { + Some(broken) => Err(broken), + None => Ok(extraction), + } +} + +/// ONT-4c5, before validation: `apply` on the gate's own graph when a contract declares `capability-cells`. A +/// non-numeral receipt version is the declaration's fault (exit 3); |D| = 0 is R-2's decline, never Pass. +fn cells_corpus( + owned: &mut Graph, + shapes: &[NodeShape], + extraction: &extract::Extraction, +) -> Result, ShapesOutcome> { + let cells = cells_gate::corpus(owned, shapes, &extraction.gguf.rungs, &extraction.receipts) + .map_err(|e| { + ShapesOutcome::ExtractFailed(ExtractFailure::Receipt(receipts::ReceiptError { + file: e.file(), + what: e.to_string(), + })) + })?; + if cells.as_ref().is_some_and(|c| c.domain.is_empty()) { + return Err(ShapesOutcome::EmptyDomain { + shapes_n: shapes.len(), + }); + } + Ok(cells) +} + +/// ONT-4c5, after validation: every per-shape positive control fired, and the corpus went through `apply` (the +/// rungs the shape flagged are the rungs owning a NotRun cell). Either miss is a decline, never a verdict. +fn cells_controls( + shapes: &[NodeShape], + arming: &ArmedShapes, + report: &Report, + rungs: &[extract::gguf::Rung], + cells: Option<&capability_cells::CapabilityCells>, +) -> Result, ShapesOutcome> { + let pc_shapes = cells_gate::pc_shapes(shapes, arming); + if let Some((which, _)) = pc_shapes.iter().find(|(_, v)| v.as_str() != "fired") { + return Err(ShapesOutcome::PositiveControlFailed { + shapes_n: shapes.len(), + focus_nodes_n: report.focus_nodes_n, + which: format!("pc_shapes.{which}"), + }); + } + if let Some(cc) = cells { + if !cells_gate::wiring_holds(report, rungs, cc) { + return Err(ShapesOutcome::Differential { + shapes_n: shapes.len(), + focus_nodes_n: report.focus_nodes_n, + passed: 0, + n: 1, + failed: vec![format!( + "capability-cells wiring: the rungs the shape flagged differ from the rungs owning the {} NotRun cell(s)", + cc.not_run.len() + )], + }); + } + } + Ok(pc_shapes) +} + /// The shape set this run grades and its arming, or the answer that stands in for a verdict (unsupported or /// malformed shapes, no shapes at all, a `--shape` naming nothing). A named family is armed whatever /// `armed_shapes` says (#3715). @@ -389,10 +516,59 @@ fn needs_receipts(shapes: &[NodeShape]) -> bool { }) } -/// Violations fail; warnings alone are `Unknown{Warn}`, never a pass; nothing is `Pass`. -fn verdict_of(counted: &Counted) -> Verdict { +/// #3610: each shape's focus-node count, in declaration order. +fn focus_of(graph: &Graph, shapes: &[NodeShape]) -> Vec<(String, usize)> { + shapes + .iter() + .map(|s| { + ( + s.id.clone(), + shapes::instances_closed(graph, &s.target_class).len(), + ) + }) + .collect() +} + +/// #3610: every shape that graded ZERO focus nodes, and whether any of them is ARMED. +/// +/// TWO answers, because they are two questions and conflating them was the defect a quorum lane +/// caught (#3610 round 1, two lanes independently). The list decides where a vacuity is REPORTED — +/// `declines` only, never `not_armed_shapes`, or an unarmed vacuity is filed as a policy choice, +/// which is how the original defect hid. The flag decides the VERDICT: only an armed shape fed it, and a +/// shape that declares `allowEmpty: ""` (a target class empty by design) is named but does not refuse. +fn vacuities( + focus_of: &[(String, usize)], + shapes: &[NodeShape], + arming: &ArmedShapes, +) -> (Vec, bool) { + let vacuous: Vec = focus_of + .iter() + .filter(|(_, n)| *n == 0) + .map(|(id, _)| id.clone()) + .collect(); + let may_be_empty = |id: &str| shapes.iter().any(|s| s.id == id && s.allow_empty.is_some()); + let armed = vacuous + .iter() + .any(|id| arming.is_armed(id) && !may_be_empty(id)); + (vacuous, armed) +} + +/// Violations fail; an ARMED shape that graded nothing is `Unknown{NoFocus}` (#3610); warnings alone are +/// `Unknown{Warn}`, never a pass; nothing is `Pass`. +/// +/// THE REFUSAL IS THE EXIT CODE; THE REPORT IS THE EVIDENCE. A vacuity declines through the ordinary +/// result path rather than short-circuiting, so stdout still carries the full JSON (`by_shape` naming the +/// zero-focus shape, `declines`) and `meet_exit` turns Unknown into exit 2 — downstream consumers (infra's +/// SLK gate) parse stdout regardless of the exit code. +/// +/// A MEASURED violation outranks a vacuity (#3622 re-review, all three lanes): violations and vacuities +/// range over DISJOINT shapes, so checking vacuity first would turn a real Fail from one shape into exit 2 +/// because a DIFFERENT shape graded nothing. +fn verdict_of(counted: &Counted, armed_vacuity: bool) -> Verdict { if counted.violations > 0 { Verdict::Fail + } else if armed_vacuity { + Verdict::Unknown(Reason::NoFocus) } else if counted.warnings > 0 { Verdict::Unknown(Reason::Warn) } else { @@ -401,56 +577,124 @@ fn verdict_of(counted: &Counted) -> Verdict { } /// `shape=focus-count` per shape, sorted. -fn by_shape(graph: &Graph, shapes: &[NodeShape]) -> Vec { - let mut out: Vec = shapes - .iter() - .map(|s| { - let n = shapes::instances_closed(graph, &s.target_class).len(); - format!("{}={}", s.id, n) - }) - .collect(); +fn by_shape(focus_of: &[(String, usize)]) -> Vec { + let mut out: Vec = focus_of.iter().map(|(id, n)| format!("{id}={n}")).collect(); out.sort(); out } -/// Focus nodes each extractor produced. -fn by_entity_type(extraction: &extract::Extraction) -> BTreeMap { - [ - ( - "pv-contract", - extraction - .graph - .instances_of(&crate::ontology::rdf::ont("Contract")) - .len(), - ), - ( - "gguf", - extraction.gguf.rungs.len() + extraction.gguf.files_read, - ), - ("apr-model", extraction.apr_model.files_read), - // ONT-4c3: registered in Σ and implemented, so it is counted here like every other entity - // type. Without this key a probe asking `by_entity_type["parity-receipt"]` reads ABSENT — - // and an absent key is not zero, so a consumer that treats it as one measures nothing and - // calls it a pass. The same shape as #3610, one map over. - ("parity-receipt", extraction.parity.records), - ("code", extraction.code.symbols), - ("lean", extraction.lean.statements), - ] - .into_iter() - .map(|(k, v)| (k.to_string(), v)) - .collect() +/// The entity types this build counts. Every name here has an arm in [`entity_count`] +/// ([`tests::every_counted_entity_type_has_a_counting_arm`]); every name Σ marks `implemented: true` must too, or +/// the gate refuses ([`by_entity_type`]). +const COUNTED_ENTITY_TYPES: &[&str] = &[ + "pv-contract", + "json", + "gguf", + "apr-model", + "parity-receipt", + "code", + "lean", + "release-evidence", + "cli-surface", +]; + +/// Focus nodes each extractor produced, for one Σ entity type. `None` is "this build has no counting arm for the +/// name" — never zero (#3624: an absent key is not zero). +fn entity_count(name: &str, extraction: &extract::Extraction) -> Option { + Some(match name { + "pv-contract" => extraction + .graph + .instances_of(&crate::ontology::rdf::ont("Contract")) + .len(), + // One per `entity: {type: json}` contract extracted. + "json" => extraction.entities_extracted.len(), + "gguf" => extraction.gguf.rungs.len() + extraction.gguf.files_read, + "apr-model" => extraction.apr_model.files_read, + // ONT-4c3: without this key a probe asking `by_entity_type["parity-receipt"]` reads ABSENT — and an + // absent key is not zero, so a consumer that treats it as one measures nothing and calls it a pass. + "parity-receipt" => extraction.parity.records, + "code" => extraction.code.symbols, + "lean" => extraction.lean.statements, + // By rule 0 when no release subject was given (an ordinary PR has none): the extractor did not run. + "release-evidence" => extraction.release.as_ref().map_or(0, |r| r.cells), + // #3777 registered cli-surface in Σ after #3624 derived these keys from it: the leaf commands the + // release's `apr surface --json` declares. By rule 0 without `--surface` (the extractor did not run). + "cli-surface" => extraction + .release + .as_ref() + .and_then(|r| r.surface.as_ref()) + .map_or(0, |s| s.leaves), + _ => return None, + }) +} + +/// The entity types `/ontology.yaml` (Σ) marks `implemented: true`. Empty when the tree has no Σ or +/// Σ does not parse — Σ's well-formedness is the `sigma` gate's verdict, not this one's. +fn sigma_implemented(contract_dir: &Path) -> BTreeSet { + std::fs::read_to_string(contract_dir.join("ontology.yaml")) + .ok() + .and_then(|text| crate::ontology::sigma::Sigma::from_yaml(&text).ok()) + .map(|sigma| { + sigma + .entity_types + .into_iter() + .filter(|e| e.implemented) + .map(|e| e.name) + .collect() + }) + .unwrap_or_default() +} + +/// Focus nodes each extractor produced, keyed by entity type: every type this build counts, and every type Σ +/// implements. #3624 — the keys were a hand-written array beside Σ, so `json` and `release-evidence` shipped +/// implemented and uncounted, and every `by_entity_type[""]` probe on them read ABSENT. A Σ-implemented type +/// with no counting arm is now the gate's refusal, named, never a missing key. `implemented: false` types are +/// omitted by rule unless this build counts them. +fn by_entity_type( + extraction: &extract::Extraction, + implemented: &BTreeSet, +) -> Result, ShapeError> { + COUNTED_ENTITY_TYPES + .iter() + .map(|s| (*s).to_string()) + .chain(implemented.iter().cloned()) + .map(|name| match entity_count(&name, extraction) { + Some(n) => Ok((name, n)), + None => Err(ShapeError::Malformed { + shape: "by_entity_type".into(), + what: format!( + "entity type {name} is registered in Σ as implemented and is not counted in by_entity_type" + ), + }), + }) + .collect() +} + +fn harness_broken(extraction: &extract::Extraction) -> Option { + let causes = extraction + .release + .as_ref()? + .crux + .as_ref()? + .harness_broken + .clone(); + (!causes.is_empty()).then_some(ShapesOutcome::HarnessBroken { causes }) } /// The answers that are not corpus verdicts, in the order they are asked: no focus node, receipts needed and /// none tracked, the plant silent, an extractor control silent. `None` when the corpus gets a verdict. +/// +/// No focus node at all is `NoFocus` unless EVERY shape in scope declares `allowEmpty` (#3610): then the run +/// reaches the per-shape path, which names each one in `declines` and does not refuse for them. fn decline( shapes_n: usize, report: &Report, + all_allow_empty: bool, unmeasured: bool, plant_violations: usize, pc_extract: &BTreeMap, ) -> Option { - if report.focus_nodes_n == 0 { + if report.focus_nodes_n == 0 && !all_allow_empty { return Some(ShapesOutcome::NoFocus { shapes_n }); } if unmeasured { @@ -492,6 +736,8 @@ fn extract_controls() -> BTreeMap { ), // aprender#3715: drawn every run, subject or not — a cell owed without a receipt stays a node ("release-evidence", release_evidence::positive_control()), + // aprender#3745 S2: a model arg read from its ROLE, never its name — drawn every run + ("cli-surface", cli_surface::positive_control()), ] .into_iter() .map(|(k, fired)| { @@ -748,12 +994,13 @@ mod tests { match &result.extra { Some(GateExtra::Shapes { by_entity_type, - pc_extract, + controls, witnesses, unarmed_violations, .. }) => { assert_eq!(by_entity_type.get("gguf"), Some(&2)); + let pc_extract = &controls.pc_extract; assert_eq!(pc_extract.get("gguf").map(String::as_str), Some("fired")); assert_eq!( pc_extract.get("apr-model").map(String::as_str), @@ -937,4 +1184,53 @@ mod tests { assert_eq!(v, "fired", "pc_extract.{k}"); } } + + #[test] + fn every_counted_entity_type_has_a_counting_arm() { + let x = extract::Extraction::default(); + for name in COUNTED_ENTITY_TYPES { + assert!( + entity_count(name, &x).is_some(), + "{name} is listed as counted and has no arm" + ); + } + } + + #[test] + fn every_implemented_entity_type_in_sigma_is_a_by_entity_type_key() { + // aprender#3624 — `json` and `release-evidence` were implemented in Σ and absent from the map, so a + // `jq -e '.by_entity_type["json"] == N'` probe read null. Read the REAL Σ the way the gate does. + let contracts = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contracts"); + let implemented = sigma_implemented(&contracts); + assert!( + implemented.contains("json") && implemented.contains("release-evidence"), + "Σ no longer implements the two #3624 types — the test would be vacuous: {implemented:?}" + ); + let map = by_entity_type(&extract::Extraction::default(), &implemented) + .expect("every implemented type has an arm"); + for name in &implemented { + assert!( + map.contains_key(name), + "by_entity_type lacks Σ-implemented {name}" + ); + } + assert_eq!( + map.get("release-evidence"), + Some(&0), + "no subject: 0 by rule" + ); + } + + #[test] + fn an_implemented_type_without_a_counting_arm_is_refused_by_name() { + // The discrimination #3624 asks for: flip a type to implemented with no arm, and the gate names it. + let flipped: BTreeSet = ["gguf", "readme"].iter().map(|s| s.to_string()).collect(); + match by_entity_type(&extract::Extraction::default(), &flipped) { + Err(ShapeError::Malformed { what, .. }) => assert!( + what.contains("entity type readme is registered in Σ as implemented"), + "{what}" + ), + other => panic!("expected a named refusal, got {other:?}"), + } + } } diff --git a/crates/aprender-contracts/src/lint/valid_under_gate.rs b/crates/aprender-contracts/src/lint/valid_under_gate.rs new file mode 100644 index 0000000000..17723cda64 --- /dev/null +++ b/crates/aprender-contracts/src/lint/valid_under_gate.rs @@ -0,0 +1,341 @@ +//! ONT-001 §5 ONT-7 — the `valid-under` gate: a kernel-kind contract carries a WORLD INDEX. +//! +//! A kernel contract's claim ("row sums to 1 within ε") holds in some world: a toolchain, a host class, +//! a backend, a feature set. Until this gate, nothing said which, so a claim measured on `cuda` read the +//! same as one that holds everywhere. `metadata.valid_under` names it: +//! +//! ```yaml +//! metadata: +//! valid_under: +//! world: committed # a key of Σ's `worlds:` (the index); omitted = `committed` +//! toolchain: { rust: "1.93" } # optional qualifiers, closed set +//! host_class: [x86_64-linux] +//! backend: [cpu, cuda] +//! features: [cuda] +//! ``` +//! +//! Rules, all `reject:` (exit 1) because the corpus is what is wrong: +//! +//! - PV-ONT-013 — `valid_under` is present but not a mapping, is empty, or carries a key outside the closed set; +//! - PV-ONT-014 — `world` is not a string, or names a world Σ does not declare (an omitted `world` reads +//! [`DEFAULT_WORLD`], which Σ must declare — so the spec's Appendix B example is admitted as written); +//! - PV-ONT-015 — a qualifier has the wrong shape (`toolchain` a map of strings; the others non-empty lists +//! of non-empty strings); +//! - PV-ONT-016 — the `contracts_without_valid_under` ratchet ROSE. The debt is the kernel-kind contracts +//! (non-registry) that carry no `valid_under`; it is recorded at the TOP LEVEL of +//! `contracts/lint-baseline.json` (the row's probe reads it there) and is shrink-only, the +//! `formal_prose` pattern. Without a baseline the count is reported, never treated as satisfied. +//! +//! The rules apply to `valid_under` wherever it appears; only the ratchet is scoped to kernel-kind, because +//! that is the class the row obliges. Non-verdict answers: no Σ → decline, malformed Σ → error, and a corpus +//! with no kernel-kind contract AND no `valid_under` anywhere measured nothing → decline (ONT R-2: zero is a +//! decline, never an accept). The gate is computed in every `pv lint` run (gate 13, R-8) and armed per repo. +//! +//! **Reads RAW YAML for the key** (the `Contract` struct does not carry it, and serde drops what it does +//! not know — sigma_gate's reason), and the PARSED contract for the kind, so "kernel" means exactly what +//! `pv validate` means by it, including the absent-kind default. + +use std::collections::BTreeMap; +use std::path::Path; +use std::time::Instant; + +use crate::ontology::sigma::{Sigma, SigmaError}; +use crate::schema::{parse_contract, ContractKind}; + +use super::finding::LintFinding; +use super::rules::RuleSeverity; +use super::{GateDetail, GateExtra, GateResult, Verdict}; + +/// The keys `metadata.valid_under` may carry. Closed world: anything else is a reject. +pub const VALID_UNDER_KEYS: [&str; 5] = ["world", "toolchain", "host_class", "backend", "features"]; + +/// The world a `valid_under` without `world:` is read in. Σ declares it, in its own words: `committed` is +/// "the world every contract is read in unless it says otherwise" (contracts/ontology.yaml). So the spec's +/// Appendix B example — qualifiers and no `world` — is admitted, and indexes `committed`. It must still be a +/// world Σ declares: a Σ without `committed` gives an omitted `world` nothing to default to (PV-ONT-014). +pub const DEFAULT_WORLD: &str = "committed"; + +/// The top-level key of `lint-baseline.json` that records the debt. +pub const BASELINE_KEY: &str = "contracts_without_valid_under"; + +/// What one `valid-under` run answers. Only [`ValidUnderOutcome::Ran`] is a verdict about the corpus. +#[derive(Debug)] +pub enum ValidUnderOutcome { + /// No `ontology.yaml` under the corpus: there is no world index to resolve against. + NoSigma, + /// Σ does not parse, or does not satisfy its own integrity rules. + Malformed(SigmaError), + /// The corpus holds no kernel-kind contract: nothing the row obliges was measured. + NoKernels { + /// Contract files read. + contracts_checked: usize, + }, + /// Σ was read and the corpus was checked against it. + Ran { + result: Box, + findings: Vec, + }, +} + +/// Run the gate over `contract_dir`, reading Σ from `/ontology.yaml`. +#[must_use] +pub fn run_valid_under_gate(contract_dir: &Path) -> ValidUnderOutcome { + let start = Instant::now(); + let sigma_path = contract_dir.join("ontology.yaml"); + let Ok(text) = std::fs::read_to_string(&sigma_path) else { + return ValidUnderOutcome::NoSigma; + }; + let sigma = match Sigma::from_yaml(&text) { + Ok(s) => s, + Err(e) => return ValidUnderOutcome::Malformed(e), + }; + if let Err(e) = sigma.check_integrity() { + return ValidUnderOutcome::Malformed(e); + } + + let mut c = Census::default(); + let mut files = Vec::new(); + super::collect_yaml_files(contract_dir, &mut files); + files.sort(); + for file in files.iter().filter(|f| **f != sigma_path) { + c.observe(&sigma, file); + } + // Zero is a decline only when NOTHING was measured: a malformed `valid_under` on a non-kernel contract is + // still a reject (the rules apply wherever the key appears — #4076 re-review, lane 2). + if c.kernels == 0 && c.carrying == 0 { + return ValidUnderOutcome::NoKernels { + contracts_checked: c.checked, + }; + } + let baseline = baseline_without_valid_under(contract_dir); + c.findings + .extend(ratchet_finding(baseline, c.kernels_without)); + let Census { + findings, + checked, + kernels, + kernels_without, + carrying, + by_world, + } = c; + + let violations = findings.len(); + let passed = violations == 0; + let duration = u64::try_from(start.elapsed().as_millis()).unwrap_or(u64::MAX); + let result = GateResult { + name: "valid-under".into(), + passed, + skipped: false, + verdict: Verdict::from_gate(passed, false), + duration_ms: duration, + // `GateDetail` is FROZEN at the 0.3.1 variants (see `GateExtra`), so the shape is borrowed as the + // `sigma` gate borrows it and the real payload rides in `GateExtra`. + detail: GateDetail::Validate { + contracts: checked, + errors: violations, + warnings: 0, + error_messages: findings.iter().map(|f| f.message.clone()).collect(), + }, + extra: Some(GateExtra::ValidUnder { + worlds: sigma.worlds.keys().cloned().collect(), + contracts_checked: checked, + kernel_contracts: kernels, + contracts_with_valid_under: carrying, + contracts_without_valid_under: kernels_without, + baseline, + by_world: by_world.iter().map(|(w, n)| format!("{w}={n}")).collect(), + violations, + }), + }; + ValidUnderOutcome::Ran { + result: Box::new(result), + findings, + } +} + +/// What one pass over the corpus counted. Split out of [`run_valid_under_gate`] under the complexity ratchet +/// (cognitive 40 → below 25); the arithmetic is unchanged. +#[derive(Default)] +struct Census { + findings: Vec, + checked: usize, + kernels: usize, + kernels_without: usize, + carrying: usize, + by_world: BTreeMap, +} + +impl Census { + /// Read one file. Not YAML, or not a contract, is the `validate` gate's business and is skipped here; + /// the kind is read through the one parser `pv validate` uses. + fn observe(&mut self, sigma: &Sigma, file: &Path) { + let Some(doc) = std::fs::read_to_string(file) + .ok() + .and_then(|raw| serde_yaml::from_str::(&raw).ok()) + else { + return; + }; + let Ok(contract) = parse_contract(file) else { + return; + }; + self.checked += 1; + let is_kernel = contract.kind() == ContractKind::Kernel && !contract.is_registry(); + self.kernels += usize::from(is_kernel); + let Some(v) = doc.get("metadata").and_then(|m| m.get("valid_under")) else { + self.kernels_without += usize::from(is_kernel); + return; + }; + self.carrying += 1; + let stem = file + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or_default() + .to_string(); + let before = self.findings.len(); + check_valid_under(sigma, v, &stem, file, &mut self.findings); + if self.findings.len() == before { + let w = v + .get("world") + .and_then(serde_yaml::Value::as_str) + .unwrap_or(DEFAULT_WORLD); + *self.by_world.entry(w.to_string()).or_default() += 1; + } + } +} + +/// PV-ONT-016: the debt may fall, never rise. No baseline → nothing to compare against, reported only. +fn ratchet_finding(baseline: Option, without: usize) -> Option { + let b = baseline?; + if without <= b { + return None; + } + let mut f = LintFinding::new( + "PV-ONT-016", + RuleSeverity::Error, + format!( + "{BASELINE_KEY} rose {b} -> {without}: a kernel-kind contract without `metadata.valid_under` was added. The baseline in contracts/lint-baseline.json is shrink-only — give the new contract a world" + ), + "contracts/lint-baseline.json".to_string(), + ); + f.contract_stem = Some("lint-baseline".to_string()); + Some(f) +} + +fn finding(rule: &str, msg: String, stem: &str, file: &Path) -> LintFinding { + let mut f = LintFinding::new(rule, RuleSeverity::Error, msg, file.display().to_string()); + f.contract_stem = Some(stem.to_string()); + f +} + +/// Every rule a present `metadata.valid_under` must satisfy. +fn check_valid_under( + sigma: &Sigma, + v: &serde_yaml::Value, + stem: &str, + file: &Path, + out: &mut Vec, +) { + let Some(map) = v.as_mapping() else { + out.push(finding( + "PV-ONT-013", + "`metadata.valid_under` must be a mapping — `world:` (omitted = committed) and/or the qualifiers toolchain, host_class, backend, features".to_string(), + stem, + file, + )); + return; + }; + for key in map.keys() { + let name = key.as_str().unwrap_or(""); + if !VALID_UNDER_KEYS.contains(&name) { + out.push(finding( + "PV-ONT-013", + format!( + "`metadata.valid_under.{name}` is not a valid_under key (closed set: {})", + VALID_UNDER_KEYS.join(", ") + ), + stem, + file, + )); + } + } + if map.is_empty() { + out.push(finding( + "PV-ONT-013", + "`metadata.valid_under` is empty — name a `world:` or a qualifier, or remove the key" + .to_string(), + stem, + file, + )); + return; + } + match map.get("world").map(|w| w.as_str()) { + None if !sigma.worlds.contains_key(DEFAULT_WORLD) => out.push(finding( + "PV-ONT-014", + format!("`metadata.valid_under` names no `world:` and Σ declares no `{DEFAULT_WORLD}` world to default to"), + stem, + file, + )), + None => {} + Some(None) => out.push(finding( + "PV-ONT-014", + "`metadata.valid_under.world` must be a string naming a world in contracts/ontology.yaml".to_string(), + stem, + file, + )), + Some(Some(w)) if !sigma.worlds.contains_key(w) => out.push(finding( + "PV-ONT-014", + format!( + "`metadata.valid_under.world: {w}` is not a world Σ declares (worlds: {}) — declare it in contracts/ontology.yaml", + sigma.worlds.keys().cloned().collect::>().join(", ") + ), + stem, + file, + )), + Some(Some(_)) => {} + } + if let Some(tc) = map.get("toolchain") { + let ok = tc.as_mapping().is_some_and(|m| { + !m.is_empty() + && m.iter().all(|(k, v)| { + k.as_str().is_some_and(|s| !s.is_empty()) + && v.as_str().is_some_and(|s| !s.is_empty()) + }) + }); + if !ok { + out.push(finding( + "PV-ONT-015", + "`metadata.valid_under.toolchain` must be a non-empty map of tool -> version strings (e.g. `{ rust: \"1.93\" }`)".to_string(), + stem, + file, + )); + } + } + for key in ["host_class", "backend", "features"] { + if let Some(list) = map.get(key) { + let ok = list.as_sequence().is_some_and(|s| { + !s.is_empty() && s.iter().all(|x| x.as_str().is_some_and(|s| !s.is_empty())) + }); + if !ok { + out.push(finding( + "PV-ONT-015", + format!("`metadata.valid_under.{key}` must be a non-empty list of non-empty strings"), + stem, + file, + )); + } + } + } +} + +/// The top-level `contracts_without_valid_under` from `/lint-baseline.json`, when recorded. +fn baseline_without_valid_under(contract_dir: &Path) -> Option { + let raw = std::fs::read_to_string(contract_dir.join("lint-baseline.json")).ok()?; + let doc: serde_json::Value = serde_json::from_str(&raw).ok()?; + doc.get(BASELINE_KEY)? + .as_u64() + .and_then(|n| usize::try_from(n).ok()) +} + +#[cfg(test)] +#[path = "valid_under_gate_tests.rs"] +mod tests; diff --git a/crates/aprender-contracts/src/lint/valid_under_gate_tests.rs b/crates/aprender-contracts/src/lint/valid_under_gate_tests.rs new file mode 100644 index 0000000000..f90b46f0ee --- /dev/null +++ b/crates/aprender-contracts/src/lint/valid_under_gate_tests.rs @@ -0,0 +1,220 @@ +//! ONT-7 — the rules of `metadata.valid_under`, one case per rule, against the real Σ. + +use super::*; + +fn sigma() -> Sigma { + let text = std::fs::read_to_string( + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../contracts/ontology.yaml"), + ) + .expect("the repo's Σ is readable"); + Sigma::from_yaml(&text).expect("the repo's Σ parses") +} + +fn rules_for(yaml: &str) -> Vec { + let v: serde_yaml::Value = serde_yaml::from_str(yaml).expect("case is YAML"); + let mut out = Vec::new(); + check_valid_under(&sigma(), &v, "case", Path::new("case.yaml"), &mut out); + out.into_iter().map(|f| f.rule_id).collect() +} + +#[test] +fn a_full_world_index_draws_nothing() { + let y = "world: committed\ntoolchain: {rust: \"1.93\"}\nhost_class: [x86_64-linux]\nbackend: [cpu, cuda]\nfeatures: [cuda]\n"; + assert!(rules_for(y).is_empty()); +} + +#[test] +fn world_alone_is_enough() { + assert!(rules_for("world: committed\n").is_empty()); +} + +#[test] +fn an_undeclared_world_is_pv_ont_014() { + assert_eq!(rules_for("world: mars\n"), ["PV-ONT-014"]); +} + +#[test] +fn a_non_string_world_is_pv_ont_014() { + assert_eq!(rules_for("world: [committed]\n"), ["PV-ONT-014"]); + // An OMITTED world is not this rule: it reads `committed` (see the Appendix B test below). + assert!(rules_for("backend: [cpu]\n").is_empty()); +} + +#[test] +fn a_key_outside_the_closed_set_is_pv_ont_013() { + assert_eq!(rules_for("world: committed\ngpu: true\n"), ["PV-ONT-013"]); +} + +#[test] +fn a_non_mapping_is_pv_ont_013_and_nothing_else() { + assert_eq!(rules_for("committed\n"), ["PV-ONT-013"]); +} + +#[test] +fn malformed_qualifiers_are_pv_ont_015_each() { + assert_eq!(rules_for("world: committed\nbackend: []\n"), ["PV-ONT-015"]); + assert_eq!( + rules_for("world: committed\nhost_class: [\"\"]\n"), + ["PV-ONT-015"] + ); + assert_eq!( + rules_for("world: committed\nfeatures: cuda\n"), + ["PV-ONT-015"] + ); + assert_eq!( + rules_for("world: committed\ntoolchain: {rust: 1}\n"), + ["PV-ONT-015"] + ); + assert_eq!( + rules_for("world: committed\ntoolchain: {}\n"), + ["PV-ONT-015"] + ); +} + +#[test] +fn the_closed_set_is_exactly_the_five_keys() { + assert_eq!( + VALID_UNDER_KEYS, + ["world", "toolchain", "host_class", "backend", "features"] + ); +} + +// ── the omitted world and the empty block (#4076 re-review) ───────────────────────────────────────────── + +#[test] +fn an_omitted_world_reads_committed_so_appendix_b_is_admitted() { + let y = "toolchain: {rust: \"1.93\"}\nhost_class: [x86_64-linux]\nbackend: [cpu, cuda]\nfeatures: [cuda]\n"; + assert!(rules_for(y).is_empty(), "the spec's own example must pass"); + assert_eq!(DEFAULT_WORLD, "committed"); +} + +#[test] +fn an_omitted_world_with_no_committed_in_sigma_is_pv_ont_014() { + let mut s = sigma(); + s.worlds.remove(DEFAULT_WORLD); + let v: serde_yaml::Value = serde_yaml::from_str("backend: [cpu]\n").expect("yaml"); + let mut out = Vec::new(); + check_valid_under(&s, &v, "case", Path::new("case.yaml"), &mut out); + assert_eq!( + out.into_iter().map(|f| f.rule_id).collect::>(), + ["PV-ONT-014"] + ); +} + +#[test] +fn an_empty_block_is_pv_ont_013() { + assert_eq!(rules_for("{}\n"), ["PV-ONT-013"]); +} + +// ── the ratchet ────────────────────────────────────────────────────────────────────────────────────────── + +#[test] +fn the_ratchet_rejects_a_rise_only() { + assert!(ratchet_finding(Some(5), 4).is_none(), "a fall passes"); + assert!(ratchet_finding(Some(5), 5).is_none(), "equal passes"); + assert_eq!( + ratchet_finding(Some(5), 6).map(|f| f.rule_id).as_deref(), + Some("PV-ONT-016") + ); + assert!( + ratchet_finding(None, 1000).is_none(), + "no baseline: reported, not judged" + ); +} + +// ── the whole gate, over the committed fixtures (the lib-level witness the CI mutation lane runs) ───────── + +fn fixture(name: &str) -> std::path::PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../tests/fixtures/ont") + .join(name) +} + +fn ran(name: &str) -> (bool, Vec, Option) { + match run_valid_under_gate(&fixture(name)) { + ValidUnderOutcome::Ran { result, findings } => ( + result.passed, + findings.into_iter().map(|f| f.rule_id).collect(), + result.extra, + ), + other => panic!("{name}: expected a verdict, got {other:?}"), + } +} + +#[test] +fn every_fixture_draws_exactly_its_rule() { + for (name, want) in [ + ("valid-under-ok", vec![]), + ("valid-under-appendix-b", vec![]), + ("valid-under-unknown-world", vec!["PV-ONT-014"]), + ("valid-under-world-not-a-string", vec!["PV-ONT-014"]), + ("valid-under-undeclared-key", vec!["PV-ONT-013"]), + ("valid-under-empty", vec!["PV-ONT-013"]), + ( + "valid-under-bad-qualifier", + vec!["PV-ONT-015", "PV-ONT-015"], + ), + ("valid-under-ratchet-rise", vec!["PV-ONT-016"]), + ("valid-under-nonkernel-bad", vec!["PV-ONT-014"]), + ] { + let (passed, rules, _) = ran(name); + assert_eq!(rules, want, "{name}"); + assert_eq!(passed, want.is_empty(), "{name}"); + } +} + +#[test] +fn the_census_counts_what_it_saw() { + let (_, _, extra) = ran("valid-under-ratchet-rise"); + let Some(GateExtra::ValidUnder { + kernel_contracts, + contracts_without_valid_under, + contracts_with_valid_under, + baseline, + .. + }) = extra + else { + panic!("valid-under extra"); + }; + assert_eq!( + ( + kernel_contracts, + contracts_without_valid_under, + contracts_with_valid_under, + baseline + ), + (1, 1, 0, Some(0)) + ); + let (_, _, extra) = ran("valid-under-appendix-b"); + let Some(GateExtra::ValidUnder { by_world, .. }) = extra else { + panic!("valid-under extra"); + }; + assert_eq!( + by_world, + ["committed=1"], + "an omitted world is counted as committed" + ); +} + +#[test] +fn a_corpus_that_measured_nothing_declines_and_no_sigma_declines() { + assert!(matches!( + run_valid_under_gate(&fixture("valid-under-no-kernels")), + ValidUnderOutcome::NoKernels { + contracts_checked: 1 + } + )); + assert!(matches!( + run_valid_under_gate(&fixture("sigma-absent")), + ValidUnderOutcome::NoSigma + )); +} + +#[test] +fn the_named_gate_dispatches_valid_under() { + assert!(super::super::NAMED_GATES.contains(&"valid-under")); + assert!(matches!( + super::super::run_named_gate(&fixture("valid-under-ok"), "valid-under"), + super::super::NamedGateOutcome::ValidUnder(ValidUnderOutcome::Ran { .. }) + )); +} diff --git a/crates/aprender-contracts/src/ontology/capability_cells.rs b/crates/aprender-contracts/src/ontology/capability_cells.rs new file mode 100644 index 0000000000..82b1c35f84 --- /dev/null +++ b/crates/aprender-contracts/src/ontology/capability_cells.rs @@ -0,0 +1,277 @@ +//! ONT-001 §5 ONT-4c5 — capability cells: a required cell of the model-capability ladder is oracle-proven or +//! RED. `Unknown{NotRun}` on a required cell never arms. +//! +//! **Domain D** — the cells ONT-4c1's `ladder-green` quantifies over, restricted to `required: true` rungs: +//! `{ "@" : rung.required, host ∈ receipts::expected_hosts(rung, receipts::receipt_hosts(all)) }`. +//! One definition: the host axis is ladder-green's own function, not a copy. +//! +//! **Cells are measured at the current release, V\*, only.** V\* is the greatest `version` over every receipt +//! `receipts::read_all` read; a version that does not parse as dotted numerals is refused (the declaration's +//! fault, never guessed). A cell is measured ONLY by a witness row (its `sha256` equals the rung's) in a V\* +//! receipt for that host. Older versions are never consulted: every required rung carries the same sha across +//! 0.68.1 / 0.68.2 / 0.69.1, so ladder-green's cross-version "∃ witness row" would let a 0.68.2 green row hide a +//! 0.69.1 DEFER or a missing 0.69.1 row (plan grill round 1, measured on this tree). +//! +//! **Per row, a label beats the measurement.** A row label (`verdict`, `label`) that `Verdict::from_label` maps to +//! `Unknown(_)` — DEFER, MANUAL, NO-VERDICT (`Unknown(NotRun)`), SKIP, REPORT, WARN (their own reason) — puts the +//! cell in `not_run` whatever its `green` says; a label +//! that maps to `Fail` makes it Fail; `Pass` labels never upgrade a row (the measurement decides); a label +//! `from_label` does not know is REFUSED by name and the row is NotRun. Otherwise Pass = `receipts::row_is_green` +//! and Fail = its negation. **Several V\* rows for one cell** combine: any NotRun → NotRun; else any Fail → Fail; +//! else Pass — a green duplicate never hides a DEFER or a Fail. +//! +//! **The set difference is the validator's.** `not_run = D \ { cells measured Pass or Fail }`. An absent cell is +//! NotRun, never folded into Fail: this row ADMITS Fail (ladder-green is what refuses it), so a folded absence +//! would pass. [`apply`] writes, on the rung's `model:Model` node, `model:capabilityCell "@="` per +//! found cell (the extractor's part: only what it finds), and `model:notRunCell "@"` per member of +//! `not_run` and `model:refusedLabel ": :