Repository navigation
core: keep exact image identities beside prompt-cache words - #1962
Conversation
|
Thanks @laudney. We reviewed this independently, and it lands. Every lookup and keep path now also requires the entry's image identities and family frame to match. For text-only prompts and families that don't pass images both are empty, so keys, lookups, saved bytes, charges and the disk format are unchanged. The identity is position, row count, grid and a 62-bit content hash over the image bytes plus grid, plus the frame, and never a path or URL. Your test's real 31-bit collision with no wrong resume is a nice touch. Image-word states are no longer written to or read from disk, or learned, for any family with image rows. That's conservative but a behaviour change, so we'll say so in the release notes. Two small points: eviction order still compares words only, which affects order but never causes a wrong resume. And please drop the overlap line in the body that names an internal tool. |
|
Thanks. I removed the overlap section from the body (and from #1965). On eviction order: agreed. It compares the compact words only, so a collision can change which entry goes first, but the identity check on every lookup and keep path means it never causes a wrong resume. I have left it as is. |
Keep image prefix identities and family frames beside cache words so colliding one-row images cannot resume each other's state.
6bda677 to
150fdda
Compare
|
Thank you @laudney. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes. |
main<|image_pad|>per row)--learn-conversationsrows) for a prompt with image rowsPart of #1927. This is the shared-core prerequisite for the LightOnOCR-3 PR, split out so it can be reviewed on its own.
Why
A cache word is a
u32. For an image row it holds 31 bits hashed from the image identity and the row. Families whose image markers carry the whole identity never depended on that word alone. A family with a bare<|image_pad|>per row does: one-row images that collide in 31 bits (about one pair in 2^31, and a birthday bound near 2^15.5 images) produced equal cache keys, and the store could resume the wrong image's state. The row word stays as a compact index hint; the exact identity now lives beside it.What changes
zig/src/core/cache_images.zig(new):Identity {at, tokens, gh, gw, hash}, the image prefix of an entry, andmatches(store, entry).prompt_cache.zig: an entry owns its image identities and frame;free()releases them. Every lookup path checksmatches: plain and user lookups,prompt_interior.zig,prompt_cache_index.zig, and the resume and fork helpers.lane_job.zig,lane_stream.zig,images.zig: the request's images and the family frame reachkeepBornFrameandlookupUserFrame.rows_read.zig,rows_write.zig,prompt_learn.zig: the disk format has no image metadata, so states with image words (diskSafe) or a frame are neither written nor read.image_key.zig: the comments say what the row word is now.Tests
image_key.zig"marker-free image row collisions cannot restore another image, including adjacent one-row images"image_reuse_test.zig"exact image metadata prevents a wrong restore even when row keys are omitted"image_reuse_test.zig"image guard: a sole image-pad marker ..."<|image_pad|>marker and no wrapper tokens expands real images and leaves typed spellings as textcache_images.zigunit testsChecks on this branch (Zig 0.17.0, Linux x86_64, on
main089a18d):zig fmt --check,zig build test,zig build test -Dhip=true(gfx1201),zig build native-hip(ReleaseFast) andzig build test-lane-api-hostall pass.