From 0f32551c6891dd9a0863d1901cc30e3ec556e866 Mon Sep 17 00:00:00 2001 From: David Stuebe <84335963+emfdavid@users.noreply.github.com> Date: Wed, 16 Sep 2026 01:41:25 +0000 Subject: [PATCH 1/2] docs(tuning): flag that the block_chunks guidance is incomplete on throughput The page presents `block_chunks` as a shuffle-and-residency knob, and every piece of sizing advice on it follows from that framing. It is also a throughput knob: a block must be co-resident to gather, so on a store whose read latency has a long tail one slow read holds the concurrency window open for the rest of the block, and wider blocks amortise that. On a pass already saturated downstream of admission, wider blocks only cost residency -- so the sign depends on the store and the box. Conditioning the advice properly needs measurements this note does not have, so this marks the gap rather than guessing at a rule: readers are told the sizing advice is necessary but not sufficient, and to measure both ends of their range. The per-regime rows at the end of the page inherit the same gap. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP --- docs/tuning.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/tuning.md b/docs/tuning.md index bd2459e..69a956e 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -53,6 +53,22 @@ The corollary for the other two axes: **`block_chunks` is a training knob** (it quality, and only `.train` shuffles — eval views are deterministic), and the **cross-epoch cache is a training knob** (single-pass scoring never reads a chunk twice). +!!! note "`block_chunks` guidance on this page is under revision" + + This page presents `block_chunks` as a shuffle-and-residency knob — the sizing advice + below, and the per-regime rows at the end, all follow from that. It also affects + **throughput**, by a mechanism the page does not yet describe: a batch may draw from any + chunk in its shuffle-block, so the whole block must be co-resident to gather, and on a + store whose read latency has a long tail one slow read can hold the concurrency window + open for the rest of the block. Wider blocks amortise that; on a pass already saturated + downstream of admission they only cost residency. + + Which way it lands is therefore a property of the store and the box, not of the loader, + and the guidance here is not yet conditioned on it. Until it is, treat the `block_chunks` + sizing advice below as **necessary but not sufficient** — correct about RAM and shuffle + quality, silent about read-latency tails — and measure both ends of your range rather + than taking a default from this page. + ## The knobs you set All of these are `InSituDataset(...)` arguments except the last, which is fixed when the From 28bf6f307b7a63dc44090f4292f4e83b20e27fad Mon Sep 17 00:00:00 2001 From: David Stuebe <84335963+emfdavid@users.noreply.github.com> Date: Wed, 16 Sep 2026 02:43:49 +0000 Subject: [PATCH 2/2] docs(architecture): the shipped crop example is vectorized, not a Python loop The scope-boundaries list said the random-crop example's crop "is a per-sample Python loop, illustrative rather than vectorized". It has not been one since #32: `examples/wb2_dataloader.py` draws every sample's window origin in one batched RNG pull and gathers through a single `sliding_window_view`, so each window copies as a block instead of walking every axis element by element. The claim mattered because of where it sits. That list is what a reader consults to decide whether patching is viable at all, and "the example is illustrative" reads as "you will have to write the fast version yourself" -- when the fast version is what ships. The real cost of whole-field reads is bytes, not Python, and the corrected sentence says so, which also stops the bullet from arguing against the very example it cites. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP --- docs/architecture.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index 9bd18d4..abb2e66 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1144,8 +1144,10 @@ pretending to be a general compute graph. and memory-optimal partial-field residency — is a reserved extension (see the axis-role contract). The shipped random-crop example (`examples/wb2_dataloader.py`) does exactly this — crops in a `batch_transform` over the whole-field cache, so the - crop re-randomizes each epoch for free — but its crop is a per-sample Python loop, - illustrative rather than vectorized. + crop re-randomizes each epoch for free — and its crop is vectorized: one batched RNG + draw for every sample's window origin, then a single `sliding_window_view` gather, so + each window copies as a block rather than walking the axes element by element. What a + whole-field read costs is bytes, not Python — the crop is cheap, the read is not. - **Not a compute framework.** No general task graph, no cross-chunk reductions on the hot path, no lazy dask-style evaluation — by design (dask on the hot path is the thing we route around). Reductions like fitting a scaler run *over the loader*