Skip to content

docs: condition the block_chunks guidance, and correct the crop example's vectorization claim - #94

Merged
emfdavid merged 2 commits into
mainfrom
docs/tuning-block-chunks-note
Sep 16, 2026
Merged

emfdavid merged 2 commits into
mainfrom
docs/tuning-block-chunks-note

Conversation

@emfdavid

@emfdavid emfdavid commented Sep 16, 2026 •

Copy link
Copy Markdown
Owner

Summary

Two docs corrections, both in the same place a reader goes to decide what this engine can
and cannot do for them.

1. docs/tuning.md presents block_chunks as a shuffle-and-residency knob only

Every piece of sizing advice on that page follows from that framing — the knob table, the
multi-epoch and inference branches, and the per-regime rows at the end.

It is also a throughput knob, by a mechanism the page does not describe. A batch may draw
from any chunk in its shuffle-block, so the whole block must be co-resident to gather
(scheduler.py:36-38). On a store whose read latency has a long tail, one slow read therefore
holds the concurrency window open for the rest of the block; wider blocks amortise that. On a
pass already saturated downstream of admission, wider blocks only cost residency.

So the sign depends on the store and the box, and the page is not wrong so much as
unconditioned. Measured both ways:

block_chunks 2 → 8
real S3, c6id.8xlarge (32 vCPU), tail-bound +27%
real GCS, n2-standard-8 (8 vCPU), saturated downstream (CPU peak 98%) −13.5%

Why a note and not a rule. Conditioning the advice properly needs measurements this PR
does not have — the mechanism is established but the threshold (when does a tail dominate?)
is not, and a rule stated without it would be as wrong for one regime as the current text is
for the other. So this marks the gap: readers are told the sizing advice is necessary but not
sufficient, what the missing axis is, and to measure both ends of their range.

docs/tuning.md:582 already says "concurrency follows block_chunks" for the GRIB regime,
which is correct there — at one tile per chunk the read-ahead permit ceiling
(2 × block_chunks × tiles_per_chunk) really does gate concurrency. The page has the permit
link in the regime where it binds and is missing the tail effect in the regime where that
binds; the note covers the second.

2. docs/architecture.md said the shipped crop example is not vectorized

The scope-boundaries list described the random-crop example's crop as "a per-sample Python
loop, illustrative rather than vectorized". It has not been one since #32 —
examples/wb2_dataloader.py:99-107 draws every sample's window origin in one batched RNG pull
and gathers through a single sliding_window_view.

That list is what a reader consults to decide whether patching is viable at all, so "the
example is illustrative" reads as "you will have to write the fast version yourself" — when
the fast version is what ships. The corrected sentence also stops the bullet from arguing
against the example it cites, and names the real cost: a whole-field read costs bytes, not
Python
. Memory-optimal patching — reading only the tiles a crop touches — remains open as
#57.

For reviewers

Docs only. No code, no behaviour change. mkdocs build --strict is clean (0 warnings).

Two claims to check:

  1. The middle paragraph of the tuning.md note — that a block must be co-resident to gather,
    and that this is what lets a single slow read hold the window. That is scheduler.py:36-38
    plus source.py:77 (read_ahead_bound), not new behaviour.
  2. That examples/wb2_dataloader.py really is vectorized now — sliding_window_view at
    line 106, batched rng.integers at 99-100.

Author attestation

  • I have reviewed every change in this PR, I can explain why each one is correct, and I
    have verified the claims made in this description.

Checklist

  • uv run ruff check src tests bench examples, uv run ruff format --check src tests bench examples, uv run mypy src bench examples and uv run pytest -q are green
    locally, and uv run --extra docs mkdocs build --strict builds with 0 warnings
  • Docstrings and API docs for any new or changed public surface — n/a, no code change
  • User-facing behavior documented in docs/*.md
  • A bullet added under ## Unreleased in CHANGELOG.md — not added. Nothing changed
    for a user upgrading: one flags an existing gap in guidance, the other corrects a
    sentence that was already false. Say if you want one anyway.
  • No load-bearing invariant is broken
  • Touches ChunkPool, the scheduler, or cross-thread readiness → n/a, no code change

🤖 Generated with Claude Code

https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP

emfdavid and others added 2 commits September 16, 2026 01:54
…roughput

The page presents `block_chunks` as a shuffle-and-residency knob, and every piece of
sizing advice on it follows from that framing. It is also a throughput knob: a block
must be co-resident to gather, so on a store whose read latency has a long tail one
slow read holds the concurrency window open for the rest of the block, and wider
blocks amortise that. On a pass already saturated downstream of admission, wider
blocks only cost residency -- so the sign depends on the store and the box.

Conditioning the advice properly needs measurements this note does not have, so this
marks the gap rather than guessing at a rule: readers are told the sizing advice is
necessary but not sufficient, and to measure both ends of their range. The per-regime
rows at the end of the page inherit the same gap.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP
…hon loop

The scope-boundaries list said the random-crop example's crop "is a per-sample
Python loop, illustrative rather than vectorized". It has not been one since #32:
`examples/wb2_dataloader.py` draws every sample's window origin in one batched RNG
pull and gathers through a single `sliding_window_view`, so each window copies as a
block instead of walking every axis element by element.

The claim mattered because of where it sits. That list is what a reader consults to
decide whether patching is viable at all, and "the example is illustrative" reads as
"you will have to write the fast version yourself" -- when the fast version is what
ships. The real cost of whole-field reads is bytes, not Python, and the corrected
sentence says so, which also stops the bullet from arguing against the very example
it cites.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP
@emfdavid emfdavid changed the title docs(tuning): flag that the block_chunks guidance is incomplete on throughput docs: condition the block_chunks guidance, and correct the crop example's vectorization claim Sep 16, 2026

@emfdavid emfdavid left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@emfdavid
emfdavid merged commit d2af246 into main Sep 16, 2026
9 checks passed
@emfdavid
emfdavid deleted the docs/tuning-block-chunks-note branch September 16, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant