Repository navigation
docs: condition the block_chunks guidance, and correct the crop example's vectorization claim - #94
Merged
Merged
Conversation
…roughput The page presents `block_chunks` as a shuffle-and-residency knob, and every piece of sizing advice on it follows from that framing. It is also a throughput knob: a block must be co-resident to gather, so on a store whose read latency has a long tail one slow read holds the concurrency window open for the rest of the block, and wider blocks amortise that. On a pass already saturated downstream of admission, wider blocks only cost residency -- so the sign depends on the store and the box. Conditioning the advice properly needs measurements this note does not have, so this marks the gap rather than guessing at a rule: readers are told the sizing advice is necessary but not sufficient, and to measure both ends of their range. The per-regime rows at the end of the page inherit the same gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP
…hon loop The scope-boundaries list said the random-crop example's crop "is a per-sample Python loop, illustrative rather than vectorized". It has not been one since #32: `examples/wb2_dataloader.py` draws every sample's window origin in one batched RNG pull and gathers through a single `sliding_window_view`, so each window copies as a block instead of walking every axis element by element. The claim mattered because of where it sits. That list is what a reader consults to decide whether patching is viable at all, and "the example is illustrative" reads as "you will have to write the fast version yourself" -- when the fast version is what ships. The real cost of whole-field reads is bytes, not Python, and the corrected sentence says so, which also stops the bullet from arguing against the very example it cites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two docs corrections, both in the same place a reader goes to decide what this engine can
and cannot do for them.
1.
docs/tuning.mdpresentsblock_chunksas a shuffle-and-residency knob onlyEvery piece of sizing advice on that page follows from that framing — the knob table, the
multi-epoch and inference branches, and the per-regime rows at the end.
It is also a throughput knob, by a mechanism the page does not describe. A batch may draw
from any chunk in its shuffle-block, so the whole block must be co-resident to gather
(
scheduler.py:36-38). On a store whose read latency has a long tail, one slow read thereforeholds the concurrency window open for the rest of the block; wider blocks amortise that. On a
pass already saturated downstream of admission, wider blocks only cost residency.
So the sign depends on the store and the box, and the page is not wrong so much as
unconditioned. Measured both ways:
block_chunks2 → 8c6id.8xlarge(32 vCPU), tail-boundn2-standard-8(8 vCPU), saturated downstream (CPU peak 98%)Why a note and not a rule. Conditioning the advice properly needs measurements this PR
does not have — the mechanism is established but the threshold (when does a tail dominate?)
is not, and a rule stated without it would be as wrong for one regime as the current text is
for the other. So this marks the gap: readers are told the sizing advice is necessary but not
sufficient, what the missing axis is, and to measure both ends of their range.
docs/tuning.md:582already says "concurrency followsblock_chunks" for the GRIB regime,which is correct there — at one tile per chunk the read-ahead permit ceiling
(
2 × block_chunks × tiles_per_chunk) really does gate concurrency. The page has the permitlink in the regime where it binds and is missing the tail effect in the regime where that
binds; the note covers the second.
2.
docs/architecture.mdsaid the shipped crop example is not vectorizedThe scope-boundaries list described the random-crop example's crop as "a per-sample Python
loop, illustrative rather than vectorized". It has not been one since #32 —
examples/wb2_dataloader.py:99-107draws every sample's window origin in one batched RNG pulland gathers through a single
sliding_window_view.That list is what a reader consults to decide whether patching is viable at all, so "the
example is illustrative" reads as "you will have to write the fast version yourself" — when
the fast version is what ships. The corrected sentence also stops the bullet from arguing
against the example it cites, and names the real cost: a whole-field read costs bytes, not
Python. Memory-optimal patching — reading only the tiles a crop touches — remains open as
#57.
For reviewers
Docs only. No code, no behaviour change.
mkdocs build --strictis clean (0 warnings).Two claims to check:
tuning.mdnote — that a block must be co-resident to gather,and that this is what lets a single slow read hold the window. That is
scheduler.py:36-38plus
source.py:77(read_ahead_bound), not new behaviour.examples/wb2_dataloader.pyreally is vectorized now —sliding_window_viewatline 106, batched
rng.integersat 99-100.Author attestation
have verified the claims made in this description.
Checklist
uv run ruff check src tests bench examples,uv run ruff format --check src tests bench examples,uv run mypy src bench examplesanduv run pytest -qare greenlocally, and
uv run --extra docs mkdocs build --strictbuilds with 0 warningsdocs/*.md## UnreleasedinCHANGELOG.md— not added. Nothing changedfor a user upgrading: one flags an existing gap in guidance, the other corrects a
sentence that was already false. Say if you want one anyway.
ChunkPool, the scheduler, or cross-thread readiness → n/a, no code change🤖 Generated with Claude Code
https://claude.ai/code/session_019waAAauPpmW4QX1VaN4vqP