Repository navigation
Conversation
… N", not a 500 (D5) Operator ruling D5 (ruling-3715-2010): "a 500 is a crash-class lie". A 27155-token turn on a dense CUDA model reached BorrowedCudaForward::reserve, failed with "the device KV cache holds 4096", and came back as HTTP 500 (h4-depcheck-0d, lambda + gx10 serve/code cells). - serving_context(): the model's context capped by the device KV cache; the borrowed serve session reports it, so max_tokens clamps to what is left. - AppState caches it at construction (PMAT-073: no read lock in the handler). - serve_context_refusal(): the pre-flight 400 in the CUDA chat, CUDA /v1/completions and /generate/stream routes, before any path takes the model, so a streaming client gets a 400, not a mid-stream error. - The direct paths map ContextLimitExceeded through generation_error_status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#3715 is the pv SHACL release-readiness shape. D5 is its serve × consumer-context child: a turn longer than the device KV returned 500. The quorum judged this diff against #3715's acceptance and failed it 3/3 on the mismatch, so the serve fix now has its own ticket and acceptance. Refs #4616, #3715 Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, DEGRADED, advisory finding) Non-author reviewer (Sonnet 5), per COP 22:16Z applying the 21:10Z committed- .minisig route. diff_patch_id cf52ad666e. Verdict DEGRADED: cuda (docs MCP OAuth-gated) and mutation (no cargo test on lambda) unreachable; pmat and agy gemini-3.1-pro-high consulted; crux not triggered. Guard rejects only B1 (unsigned) until the CI signer's .minisig is committed. Advisory finding (measured): /generate and /batch/generate still map every CUDA error to 500 (batch_processing.rs:492, batch.rs:515). Follow-up, not folded here: changing the diff would void this receipt. Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…receipt Per COP 22:16Z (21:10Z committed-.minisig route). The .minisig is verbatim from pr-review-signature check-run 109170961010 (CI signer, repository key). minisign -V under .github/pr-review.pub: verified, trusted comment pr=4614 pid=cf52ad666e. check_pr_review_receipt.sh: ACCEPT (positive controls fired). Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g the serial prefill (#3715) The #3715 lambda cells timed out or failed at the 20k rung on two causes, both measured on the RTX 4090 with apr 0.70.0 (8d021f6): - Qwen3-1.7B (QK-norm): `[PREFILL-PATH] serial cc=89 (qk-norm model)`. One forward_gpu_resident per prompt token: 7955 tokens took 109 s, 26131 did not finish in 300 s at GPU util 99-100 %. Not #4609: the backend was the GPU throughout. - Qwen2.5-0.5B IQ3_M (batched): CUDA_ERROR_OUT_OF_MEMORY in prefill_all_layers_gpu at 26148 tokens. One pass allocates a num_heads x S x S f32 score matrix (38 GB), and its u32 row offsets wrap past 2^32. The batched prefill now runs in chunks whose score matrix fits the 1 GiB budget the Qwen3.5 prefill already uses; each chunk appends to the KV cache through the existing cache_len > 0 attention path. A prompt that fits one chunk runs exactly as before. QK-norm models keep FP8 off and now take the FP16 batched prefill, which #3413 measured passing CPU parity; serial was chosen only because the unchunked prefill OOMed 8B. Refs: #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3715) apr chat built its CUDA model with a flat 2048-position KV. The dense session refuses a longer turn on the device, so a 26k-token chat turn on Qwen3-1.7B ran on the CPU and hit the 600 s cell timeout (GPU util 0%, measured on lambda). apr run already sizes its KV to the turn (#4268); chat loads before it has read a turn, so it now takes the model context, capped by free VRAM after the weights, the FP16/FP8 prefill cache, the GH-178 reserve and the chunked-prefill score budget. When nothing is left (Qwen3-8B on 24 GB), it keeps the 2048 floor. Refs: #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3715) The Qwen3.5 batched prefill dequantizes each weight to F32 for one SGEMM, and had kernels for Q4_K/Q5_K/Q6_K/Q8_0 only. Qwen3.5-4B-UD-Q4_K_XL (F16 ssm_alpha/beta, IQ4_XS FFN blocks) and Qwen3.5-0.8B-IQ4_XS therefore refused the CUDA prefill, the F2 guard rejected the GPU, and every lambda cell ran on the CPU (GPU 0%) into the 600 s timeout. Adds F16DequantKernel and Iq4XsDequantKernel. The values match the per-token GEMV weights bit for bit (device tests against the ggml reader). Refs #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ched prefill does not fit (#3715) 3c2a354 moved QK-norm models from the serial prefill to the chunked FP16 batched one. The FP16 weight cache is warmed on either path, but only batched also needs the prefill workspace. Qwen3-8B on the 24 GB 4090 has 4.7 GB of weights and a 15.1 GB FP16 cache, which leaves 0.1 GB, so init_prefill_workspace failed and a 2k chat turn fell back to the CPU (382 s, rg5 C-q38b-chat-2k). The path is now decided against the device before the cache is warmed: weights + cache + KV + prefill reserve over free VRAM keeps serial, as on the base. Qwen3-1.7B (about 9 GB of 23) stays batched. BATCHED_PREFILL still overrides. The resident-bytes estimate is shared with session_kv_len. Refs: #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mutation unreachable) Reviewer agent:claude-sonnet-5/pr-review-4620; author claude-opus-5-5/aprender-ec. Written without diff_patch_id or .minisig. The CI signer stamps both. Refs: #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…they called (#4621) The last loop in fit_check_leaves_non_qk_norm_fp8_and_forced_paths_alone built two fresh profiles and asserted they were batched, so a mutant that switched the path to Serial while still returning false passed it. Each case now asserts the path of the profile it called. Red on that mutant, green on the code (intel). Quorum finding, PMAT-4621 round 1. Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uda_backend no longer grows x86-main on 099d07d was red on this diff, not on the runner: - complexity: try_cuda_backend grew (cog 43->46, cyc 14->15). The chat pre-flight now rides the tokenize result through fit_serving_context, so the function gains no branch. - mutants: serving_context lived in the cuda-gated dense_session_borrowed, where no CPU lib test can reach it. The rule is now dense_session::cap_context(context_length, device_kv), CUDA-free, with lib tests on every arm; fit_serving_context has lib tests for pass, refuse (400) and a kept tokenize error. - roadmap.yaml regenerated for the PMAT-4616 fragment. - surface_audit.csv: the /v1/completions row's cited line moved 398 -> 405. Refs #4616 Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mutation sweep unreachable; targeted mutant killed) Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… tree
Its findings.sarif carries a placeholder excerpt_sha256 ("n/a-short-code-idiom-not-hashed")
that S1.1 would reject once signed, and the receipt binds that sarif by sha256, so the
author cannot correct it without editing the reviewer's record. It is superseded by the
65370ad receipt (the head's code) and stays in history at 18b6c83. Quorum finding,
PMAT-4621 round 2 (gemini lane).
Agent: aprender-ec
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…igh, haiku-4-5) at 2b8d79a Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… 4 computes (b3f1fd942a) Arm 4 A2 reads predicate.diff_patch_id from the COMMITTED receipt; the CI signer stamps only its runner copy, so an unstamped receipt is legacy and RED. Computed with scripts/lib/pr_review_patch_id.sh prpid_compute over 722fd16..HEAD (evidence and quorum json excluded), equal to the value Arm 4 printed in run 36510383864. Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ceeded to 400 Quorum lane 2 (gemini-3.1-pro-high) at 926edb2 found two GPU-resident generate paths that still hardcoded 500 for every generation error, bypassing generation_error_status(): batch_processing.rs try_cuda_generate and batch.rs try_cuda_batch_generate. Both now route the status through it, so an over-length request is a 400 on the CUDA path as on the CPU path. cuda clippy on intel: only the pre-existing residual.rs:314/343 borrow_deref_ref. Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he signature arm Arm 4 (run 36511265641) verified the signature, then rejected the content: the antigravity divergence counted 1 against 0 findings. The reviewer corrected it to the agy output it actually recorded (findings: []), added the pmat duplication fields the guard requires, renamed two SARIF drivers to the allowed set, and re-bound findings_ref. The guard ACCEPTs it under a throwaway key. diff_patch_id is unchanged. Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… through generation_error_status Quorum at 1ead14b (sonnet-5 + gemini-3.1-pro-high FAIL) found more handlers that hardcoded 500 on a generation error, so ContextLimitExceeded still leaked as 500: SafeTensors CUDA chat, GPU /generate, logprobs, perplexity, the APR/demo batch + stream paths and the openai GPU fallback. Swept all 12 generate-error sites in api/ (decode errors stay 500: those are server faults). generation_error_status keeps every non-context error at 500, so only the over-length case changes. intel: clippy --lib default rc0; --features cuda only the pre-existing residual.rs:314/343 borrow_deref_ref; 15 targeted lib tests pass. Local mutants not_measured (unmutated baseline timed out at 600s). Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…6/IQ4_XS device dequant) The diff held 97 mutants against the gate's cap of 60. The QK-norm fit check (root 2) and the chat session KV (root 3) move to their own PR, per the cop's split ruling; they are restored here to the merge base, unchanged in content. Refs #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (root 2) Qwen3-family models were forced onto the serial prefill by disable_fp8_for_qk_norm (#3413/#3483); serial scales super-linearly and the 1.7B 20k cell ran past 600 s. They now keep FP8 off but take the FP16 batched prefill, which #4620 made chunked. A model whose weights + FP16 cache + KV + prefill reserve exceed free VRAM keeps serial (Qwen3-8B on 24 GB, which OOMed into a CPU fallback without this check); BATCHED_PREFILL=1 still overrides. The resident estimate walks the projection tensors themselves (bytes and in x out elements) instead of a dims formula, so it cannot drift from the model. Stacked on #4620. Refs #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
apr chat built its CUDA model with OwnedQuantizedModelCuda::new, a flat 2048 positions, so a 26k-token turn was refused on the device, ran on the CPU and hit the 600 s cell timeout. for_session gives the model's whole context when free VRAM holds it beside the weights, the prefill cache and the reserve, else the old 2048 floor. Stacked on the root-2 PR. Refs #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, DEGRADED, advisory findings) Non-author reviewer (Sonnet 5). Verdict DEGRADED: cuda (docs MCP OAuth-gated) and mutation (no cargo test on lambda) unreachable. Confirms the 167bd5e receipt's finding is fixed (try_cuda_generate / try_cuda_batch_generate now route through generation_error_status). Guard rejects only B1 (unsigned) until the CI signer's .minisig is committed. Advisory follow-ups, not folded (changing the diff voids this receipt): batch_processing.rs:343 batch_generate_gpu still 500; /generate and /batch/generate have no serve_context_refusal pre-flight; the effective_max_tokens `>` vs serve_context_refusal `>=` boundary. Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…7e2f The reviewer (sonnet-5) had left a placeholder in predicate.diff_patch_id; it now carries the pinned prpid value, equal to the one CI's Arm 4 measured (base 2817c6d). Guard: only B1 (unsigned) until the signer's .minisig. Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guard-complete The signature exposed a B1 the unsigned state had hidden: crux was `consulted` without surfaces/comparative_claims. Reviewer (sonnet-5) filled the S3.C arrays honestly and added the missing citation fields; verified ACCEPT under a throwaway key. patch-id binding unchanged (2fe3407e2f). Agent: aprender-2e-serve Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… graph PREFILL_GRAPH=1 (opt-in, off by default) replays a graph that bakes positions 0..S-1 and resets the KV lengths to 0. With the chunked dense prefill, chunk 2+ of the same length would have replayed it over a filled KV cache, silently corrupting it. Only a chunk starting at position 0 may take the graph now; the rest run eager. Found by the non-author Arm-4 review of #4620. Refs #4621, Refs #3715 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s 2+3, now in the stacked PRs) Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-4621) Non-author Sonnet 5 review of part A (chunked dense prefill, F16/IQ4_XS device dequant, PREFILL_GRAPH first-chunk-only). Verdict DEGRADED; diff_patch_id stamped. CI signer adds the .minisig. Refs #4621 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nto B Agent: aprender-e6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-6b Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…S4.2-compliant receipt SARIF (PMAT-4621) The quorum json is replaced with the round at head 6562916, which covers part A only (the pre-split file claimed root-3 chat KV, now C's). The receipt SARIF had its runs moved from driver 'pr-review-primary' to one run per consulted consultation, and findings_ref is rehashed. Same subject and patch-id. Refs #4621 Agent: aprender-ec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-e6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-6b Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…4096 (#4603) `apr serve` wrote its 4096 `--context-length` default to REALIZR_CONTEXT_LENGTH unconditionally, so every GGUF CUDA server built a 4096-position KV whatever the model's context (Qwen3: 40960), and the APR/SafeTensors CUDA paths used `OwnedQuantizedModelCuda::new`, a fixed 2048. A > 4096-token prompt then overflowed the KV and answered 500. - serve::run writes REALIZR_CONTEXT_LENGTH only for an operator-set flag. - serving_cuda_model (all three CUDA serve paths): an explicit length stays Fixed; otherwise `OwnedQuantizedModelCuda::for_serving` sizes the KV to min(model context, free VRAM after weights + prefill cache + reserve), each position also paying for 2 batched-KV slots, floored at the old 4096 so no model serves less than before (session_kv::serving_kv_len). Builds on #4623 (for_session / session_kv_len). The request-side 4xx for a prompt past the device KV is #4614. Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t 4xx, no second path Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, editing none of its lines R2: #4623's hunks stay as they are; K1 only adds the FitServing arm and its sizer. Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#4603) A cuda-only aprender-serve lib test, so yoga's cuda-unit job runs it with no model file. An 8192-context llama-shaped model (above serve's old fixed 4096) is sized by `for_serving` and served through the real router and the continuous-batching scheduler: - the device KV is 8192, the model's context; - /v1/completions with 8188 prompt tokens + max_tokens 4 answers 200 with all 4 tokens (usage 8188/4, finish "length"); - 8192 and 8193 prompt tokens answer 400 "context exceeds 8192 ... refused whole, not truncated", not a 500 and not a truncated 200. The weights are centred Q4_K with an FFN norm and separate Q/K/V. The shared fixture's all-positive d=1.0 pattern (cosine 0.00) and its missing FFN norm (cosine 0.54) both fail the CPU/GPU parity gate, correctly. Without a device it prints "CUDA model init failed", which cuda-unit counts as a failure, not a pass. Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…with a 400 (K1 #4603, D5 #4614) KV-only vehicle for #4630, folded (moved, not copied) from the ec stack onto main. Three rows: - #4603 K1: `apr serve` builds the CUDA model with `for_serving`, which sizes the device KV to the model's context capped by free VRAM (two batched slots budgeted, never below the old 4096). Before, serve fixed it at 4096 whatever the model declared, so a longer prompt overflowed the cache and answered 500. - #4614 / PMAT-4616 D5: a prompt at or past the serving context is refused whole with 400 "context exceeds N", on chat, completions and stream. - The KV budget core it needs (session_kv.rs, resident.rs). C25 test: api/tests/serve_kv_limit_k1.rs builds a synthetic 8192-context model, serves it through the real router and batch scheduler on the device and asserts KV == 8192, 8188+4 -> 200 with all 4 tokens, 8192 and 8193 -> 400 "refused whole". A return to Fixed(4096) fails the KV assertion. gx10 (cc 121): aprender-serve --features cuda lib filter serve_kv_limit_k1|serve_context_d5|session_kv|resident 28/28, apr-cli serve 429/429. Refs #4603 #4614 #4630 Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…op 11:03Z) The tree is exactly devont/k1-kv-only (origin/main + K1 #4603, D5 #4614 and the KV budget core). The other stack rows (#3715 serial prefill / for_session, #3413 message, #4620, #4623, #4624) stay on their ec parents for a later PR; nothing is deleted there. A merge, not a rebase or force-push. Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… 4096; the K1 test must go red. DO NOT MERGE Agent: aprender-00 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C25 revert-red probe for #4630. This is #4630's head (5d983db) with one change:
for_servinggoes back to the pre-K1 fixed 4096 KV.api::tests::serve_kv_limit_k1in cuda-unit must go red here. Its run ID is the third C25 item. Never merge; it will be closed once the red run is recorded (the branch is kept).🤖 Generated with Claude Code