Repository navigation
apr code eval E7: receipts + nightly CPU/GPU run via the fast-verb matrix (#5254) #5288
Closed as not planned
Description
A receipt JSON per run: task-set digest, model sha256, apr version + git sha, sandbox image digest, seed, per-task verdict and the metric with its CI. LAB nightly on CPU and GPU on the ARM64 CUDA runner through the #5254 verb matrix, plus one ranked gap comment. Never on a PR, merge-queue or release path.
Acceptance: two consecutive nights produce receipts, and the comment ranks tasks by failure.
Part of #5281. Spec: docs/specifications/apr-code-eval-v1.md (APR-CODE-EVAL-001).
keep-open: child of #5281
Activity
noahgift commentedon Oct 14, 2026
merged into #5314 (master ticket for epic #4000; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5342 (apr code eval: task suite, sandbox, metrics and nightly receipts (APR-CODE-EVAL-001)), split out of #5314 by theme.
chore(contracts): regenerate contracts.nt and the pv-sat witness for …
fix(code-eval): the sandbox wrapper copied the task with cp -a, which…
feat(code-eval): the smoke suite — 8 hand-written tasks, 2 per kind, …
chore(contracts): regenerate contracts.nt and the pv-sat witness for …