Skip to content

core: keep exact image identities beside prompt-cache words - #1962

Merged
ashhart merged 2 commits into
ashhart:mainfrom
laudney:pr-ocr-cache-image-ids
Nov 3, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
laudney:pr-ocr-cache-image-ids

Conversation

@laudney

@laudney laudney commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor
Case main This PR
Two different one-row images whose 31-bit row words collide, in a family whose image markers do not carry the image identity (LightOnOCR-3 1B: a sole <|image_pad|> per row) The second request can resume the first image's saved state, so its reply can differ from a fresh server The entry keeps each image's exact identity (position, rows, grid, 62-bit hash) and the family frame; the lookup refuses the state, reuses only the text before the image, and the reply equals a fresh server
Two images, only the second one changes — Resumes up to the first, unchanged image and recomputes the rest
The same image again, or text turns after an image Resumes Resumes (unchanged)
Text-only prompts Unchanged Unchanged
Disk tier (--learn-conversations rows) for a prompt with image rows The state could be written and read back without any image identity Image states stay in memory; text-only states still go to disk

Part of #1927. This is the shared-core prerequisite for the LightOnOCR-3 PR, split out so it can be reviewed on its own.

Why

A cache word is a u32. For an image row it holds 31 bits hashed from the image identity and the row. Families whose image markers carry the whole identity never depended on that word alone. A family with a bare <|image_pad|> per row does: one-row images that collide in 31 bits (about one pair in 2^31, and a birthday bound near 2^15.5 images) produced equal cache keys, and the store could resume the wrong image's state. The row word stays as a compact index hint; the exact identity now lives beside it.

What changes

  • zig/src/core/cache_images.zig (new): Identity {at, tokens, gh, gw, hash}, the image prefix of an entry, and matches(store, entry).
  • prompt_cache.zig: an entry owns its image identities and frame; free() releases them. Every lookup path checks matches: plain and user lookups, prompt_interior.zig, prompt_cache_index.zig, and the resume and fork helpers.
  • lane_job.zig, lane_stream.zig, images.zig: the request's images and the family frame reach keepBornFrame and lookupUserFrame.
  • rows_read.zig, rows_write.zig, prompt_learn.zig: the disk format has no image metadata, so states with image words (diskSafe) or a frame are neither written nor read.
  • image_key.zig: the comments say what the row word is now.

Tests

Test Shows
image_key.zig "marker-free image row collisions cannot restore another image, including adjacent one-row images" It finds a real 31-bit collision pair, builds equal cache keys from two different images, and checks that the second image does not resume the first. Text before the image still resumes, both identities keep their own state, and with two images only the unchanged first one resumes
image_reuse_test.zig "exact image metadata prevents a wrong restore even when row keys are omitted" The old planted control (no image keys, so the wrong state resumed) now feeds all 12 rows and equals a fresh server
image_reuse_test.zig "image guard: a sole image-pad marker ..." A family with one <|image_pad|> marker and no wrapper tokens expands real images and leaves typed spellings as text
cache_images.zig unit tests Prefix identity includes geometry, keeps text prefixes, and ignores later images

Checks on this branch (Zig 0.17.0, Linux x86_64, on main 089a18d): zig fmt --check, zig build test, zig build test -Dhip=true (gfx1201), zig build native-hip (ReleaseFast) and zig build test-lane-api-host all pass.

@ashhart

ashhart commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Thanks @laudney. We reviewed this independently, and it lands. Every lookup and keep path now also requires the entry's image identities and family frame to match. For text-only prompts and families that don't pass images both are empty, so keys, lookups, saved bytes, charges and the disk format are unchanged. The identity is position, row count, grid and a 62-bit content hash over the image bytes plus grid, plus the frame, and never a path or URL. Your test's real 31-bit collision with no wrong resume is a nice touch. Image-word states are no longer written to or read from disk, or learned, for any family with image rows. That's conservative but a behaviour change, so we'll say so in the release notes. Two small points: eviction order still compares words only, which affects order but never causes a wrong resume. And please drop the overlap line in the body that names an internal tool.

@laudney

laudney commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. I removed the overlap section from the body (and from #1965). On eviction order: agreed. It compares the compact words only, so a collision can change which entry goes first, but the identity check on every lookup and keep path means it never causes a wrong resume. I have left it as is.

laudney and others added 2 commits November 2, 2026 22:51
Keep image prefix identities and family frames beside cache words
so colliding one-row images cannot resume each other's state.
@ashhart ashhart mentioned this pull request Nov 3, 2026
6 of 9 tasks
@ashhart
ashhart force-pushed the pr-ocr-cache-image-ids branch from 6bda677 to 150fdda Compare November 3, 2026 00:37
@ashhart
ashhart merged commit 2c82d4f into ashhart:main Nov 3, 2026
4 of 7 checks passed
@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thank you @laudney. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Landed as 2c82d4f. Thanks @laudney!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants