Repository navigation
--workload is a free-text label decoupled from the prompts sent: a 1-prompt run is recorded as W1 #2756
Description
Found by PERF-045 (#2706).
--workload is a free-text label with no connection to the prompts actually sent.
The run passed --workload W1 --profile short. short is one prompt sent 30 times. The receipt records "workload": "W1".
Workload::from_str only checks the string is W1 or W2. The line profile short (1 prompt(s)) exists on stdout only.
The real W1 corpus is in-tree at crates/aprender-serve/benchmarks/qwen-coder/prompts-w1.jsonl — 256 distinct prompts — and its own _meta.distinctness_rationale says why that matters:
Identical prompts would let prefix caching, not the scheduler, drive Arm A's scaling_efficiency.
So the harness labels as W1 exactly the degenerate single-prompt run the corpus author documented as invalidating the measurement it feeds. An Arm A scaling number taken this way measures the prefix cache, and the receipt gives a reader no way to tell.
Suggested: bind the label to the corpus. Record a digest of the prompt set actually sent (the corpus already carries a sha), and refuse — or at minimum refuse to call it W1 — when the sent set does not match the declared workload. A label the receipt cannot falsify is not provenance.
Activity
noahgift commentedon Sep 5, 2026
PP-066 triage 2026-09-05: in 0.66 scope, no row yet — row
I-25proposed on #2873 (Still live on origin/main: no corpus digest is written into the band receipt; prompts.rs enforces the corpus's own _meta (#2778) but nothing ties the receipt's workload label to what was loaded.)Instance of class #2876 — gate answers empty/degenerate input with a PASS or a crash instead of a named refusal.
PP-066 triage 2026-09-05 (issue #2873); ledger
docs/audits/pp-066-triage-2026-09-05.md.fix(I-25): --workload is bound to the prompt corpus — receipt.workloa…
noahgift commentedon Sep 23, 2026
triage 2026-09-23 (aprender-dd, standing triage owner): milestoned to 0.73.0, theme llama.cpp Parity (performance, kernels, parity benches); it moved out of
backlog(the 2026-09-17 overflow bucket, not a release).noahgift commentedon Sep 24, 2026
Closed by paiml-implement — fixed by #3008 (merged to main) (triage evidence/triage-1418.md)