[laudney]

Lane core, readout requests (Radeon AI PRO R9700)	Before	After
A prompt-only request that reads its rows (a decision)	not a request kind	readout requests: prompt rows in pieces under the shared fill policy, then a family completion
Readout rows alone, 4 at once beside a decoding stream, after a cancel (pieces of 1,024)	—	byte-identical in all three, at 16,384 rows (16 pieces) and at 3,000 rows (3 pieces)
Two short embeddings on a prefill_many backend	batched admission took both into one addStreams pass and never called their completion, so neither finished	every prompt-only job goes through launch, which fills and completes it (fix(lanes), first commit; a fake reproduces the hang)
Other routes on a readout engine	—	refused, naming POST /v1/systemone

Stacked on #2046 (the BF16 backbone). Part of #1360. This answers the #1361 review ask that decision prompts join the shared rounds instead of a side path.

What

• fix(lanes): launch prompt-only jobs alone: on a backend with prefill_many and many_rows above zero, batched admission took two short embeddings into one addStreams pass and delivered them without their completion, so neither request finished. It can land on its own.
• feat(lanes): add readout requests: a readout Spec on the request (question and option spans), validated in the lane core, admitted like embeddings, filled in max_rows pieces under fill.run, cancelled like any job, completed by the family when the last piece is in.
• feat(server): share the prompt-only wait: embeddings and readouts share one submit-and-wait helper.
• feat(qwen3_5) / feat(hip): the HIP Readout interface (open / consume / finish / close), readout prompts through the shared prefill step, and the engine's readouts in its Info (text_generation is off when they are on).
• test(qwen35): the gate 2a readout rows check (alone, together, after a cancel).

Only a fake Readout is wired here; the joint schema head that implements it comes in the next PR.

Checks: zig build test -Dhip=true on gfx1201 (ROCm 10.1), zig fmt --check, lean checker at the baseline.

[ashhart]

Thanks @laudney. This is a clean middle layer. Every shared lane change is gated on the request kind, so text generation on every existing family takes the same path as before. The HIP Info test that checks every registered family is a nice touch.

Before this lands, could you:

1. Post the device output behind the "byte-identical in all three" row. That's the exact tf-qwen35-test readout command, the card and ROCm version, the PASS line, and the zig build test -Dhip=true summary.
2. Pin a readout's piece size in reach (always the 1,024-row chunk unless the test cut is set). Add a host test for it, and add a device run with the serving rule, alone and beside others. rocBLAS picks its BF16 kernel by row count, so today's 1,024 rests on event ordering, and a row's bits could move with it. Readouts never resuming from a kept state is already right and tested. Thanks for that.
3. Match the PR text's description of the embedding hang to commit 6 (batched admission on a prefill_many backend).

The embedding fix in the first commit can land on its own whenever you like.

[laudney]

Ask	Answer
1. Device output behind "byte-identical in all three"	below: R9700 (gfx1201), ROCm 10.1.0, hipcc HIP 7.16; PASS, rows wyhash f712c7554f2127dc alone, 4 together and after a cancel; zig build test -Dhip=true: 526/526 steps
2. Pin a readout's pieces in reach	5b9fa01 fix(lanes): pin a readout's pieces to the prompt chunk: a readout keeps the 1,024-row chunk unless the test cut is set; host test in hip_prompt_test.zig (a plain prompt beside a decoding lane gets fill_rows, a readout gets the chunk). The device check now runs the serving rule (--cut 0) beside a decoding stream, and with the pin removed it fails (control below), so the pin is what keeps the bits
3. The embedding hang text	the PR text now says what the commit fixes: batched admission on a prefill_many backend took two short embeddings into one addStreams pass and never called their completion

Device receipt

# PR #2047: readout pieces pinned to the prompt chunk (fix(lanes): pin a readout's pieces to the prompt chunk)
Card: AMD Radeon AI PRO R9700 (gfx1201). ROCm 10.1.0, hipcc HIP 7.16.26385 (core/bin). Build: zig build tf-qwen35-test -Dhip=true -Dhip-arch=gfx1201.

## The serving rule (no test cut): 1 alone, 4 together beside a decoding stream, 1 cancelled after its first piece, 1 after
$ tf-qwen35-test readout <Cloudflare/clef-flash@fb4417d3> --len 3000 --cut 0 --streams 4
policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots 
info: HIP memory budget from the card's memory: 12.9 of 29.9 GiB free, 1.0 GiB reserve
readout gate 2a: 3000 rows x 8192 bytes, the serving rule, piece 1024, pieces { { 0, 1024 }, { 1024, 1024 }, { 2048, 952 } }
PASS 1 vs 4 together beside a decoding stream and after a cancel, the serving rule: rows byte-identical (wyhash f712c7554f2127dc), 20 cuts recorded
rc=0

## The same with the test cut 1024 (the earlier gate 2a run, now beside a decoding stream too)
$ tf-qwen35-test readout <Cloudflare/clef-flash@fb4417d3> --len 3000 --cut 1024 --streams 4
policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots 
info: HIP memory budget from the card's memory: 12.9 of 29.9 GiB free, 1.0 GiB reserve
readout gate 2a: 3000 rows x 8192 bytes, a test cut, piece 1024, pieces { { 0, 1024 }, { 1024, 1024 }, { 2048, 952 } }
PASS 1 vs 4 together beside a decoding stream and after a cancel, a test cut: rows byte-identical (wyhash f712c7554f2127dc), 20 cuts recorded
rc=0

## Control: the serving rule with the pin removed (a readout follows the decoding rule): it fails
$ tf-qwen35-test readout <Cloudflare/clef-flash@fb4417d3> --len 3000 --cut 0 --streams 4
policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots 
info: HIP memory budget from the card's memory: 12.9 of 29.9 GiB free, 1.0 GiB reserve
readout gate 2a: 3000 rows x 8192 bytes, the serving rule, piece 1024, pieces { { 0, 1024 }, { 1024, 1024 }, { 2048, 952 } }
FAIL readout 1 of 4 together beside a decoding stream differs from the one alone
FAIL readout 2 of 4 together beside a decoding stream differs from the one alone
FAIL readout 3 of 4 together beside a decoding stream differs from the one alone
FAIL readout 4 of 4 together beside a decoding stream differs from the one alone
FAIL RowsDiffer
rc=1

## zig build test -Dhip=true --summary line
Build Summary: 526/526 steps succeeded; 288/294 tests passed (6 skipped)
(zig counts only the test binaries it ran again; the run before this one, with one compile error, rebuilt the suite: 6869/6918 tests passed, 49 skipped)

[ashhart]

Thanks @laudney, that's everything we asked for. The pin in reach plus the control run, where the together readouts differ once the pin is removed, show the pin is what keeps the bits. Moving the rocBLAS workspace calls to the BF16 path also answers the loader point on #2046.

One small thing: the body's table says "16K prompt", but the posted runs are 3,000 rows. Could you say which length the row covers, or add the 16K run? We don't have an AMD GPU, so your gfx1201 readout check and control are the device evidence. This can land after #2046.

[laudney]

The table row now names both lengths. The 16K run, on the same build as the receipt above (branch tip 2f0d57326, R9700 gfx1201, ROCm 10.1):

$ tf-qwen35-test readout <Cloudflare/clef-flash@fb4417d3> --len 16384 --cut 0 --streams 4
policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots 
info: HIP memory budget from the card's memory: 12.9 of 29.9 GiB free, 1.0 GiB reserve
readout gate 2a: 16384 rows x 8192 bytes, the serving rule, piece 1024, pieces { { 0, 1024 }, { 1024, 1024 }, { 2048, 1024 }, { 3072, 1024 }, { 4096, 1024 }, { 5120, 1024 }, { 6144, 1024 }, { 7168, 1024 }, { 8192, 1024 }, { 9216, 1024 }, { 10240, 1024 }, { 11264, 1024 }, { 12288, 1024 }, { 13312, 1024 }, { 14336, 1024 }, { 15360, 1024 } }
PASS 1 vs 4 together beside a decoding stream and after a cancel, the serving rule: rows byte-identical (wyhash 5492e4d9425b3977), 98 cuts recorded
rc=0

[laudney]

Merged main in cf13b0b: main's loaded-group receipts (100363c) set benchmark_ranks in the HIP lane host's Info, which this branch builds in hostInfo, so hostInfo now takes the group's world size and the Info test also checks two ranks. Lean against main: 0 new; zig build test -Dhip=true on the R9700 (ROCm 10.1): 531/531 steps.

[ashhart]

Thanks @laudney. The 16K run and the table naming both lengths close the last point, and the rocBLAS workspace commit now sits in #2046 where it belongs. Your gfx1201 receipts are the device evidence here, since we have no AMD GPU. One small thing for the rebase onto #2046: the hostInfo change from merging main lives in the merge commit, so could you rebase rather than merge, so the series itself carries it? Then this lands right after #2046.

[laudney]

Rebased onto main (ae9def8a5) as asked: 9 commits, with no merge commit and without the rocBLAS commit, which landed in #2046. Main's Serves now names what a family serves, so feat(hip): report a family's readouts in the native Info adds readouts there (text generation off when set) and drops hostInfo; main's benchmark_ranks stays in its one Info literal. The prompt-only refusal in routes.zig sits after main's reranker refusal. Checks on the R9700 (ROCm 10.1): zig build test -Dhip=true 551/551 steps; lean 0 new; tf-qwen35-test readout --len 16384 --cut 0 --streams 4 PASS with the same rows wyhash as before the rebase (5492e4d9425b3977).

[ashhart]

Thank you @laudney. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.