Repository navigation
engine api: an optional image capability beside speech (#1573) - #2248
devopsfortunato wants to merge 1 commit into
Conversation
250a1e8 to
8b703f8
Compare
|
Thanks @devopsfortunato. This is the engine half of #1573 in the shape we agreed: optional, defaulted, inert until a host sets it, and the caller-owned PNG with its allocator tested for leaks. Two things: main has moved under |
8b703f8 to
e5ca1ce
Compare
|
Rebased on current main ( Re-ran the suite on the new base: On the second point: I have asked @BobClawblaw on #1573 whether they are taking the route and the MOSS-Image host, and said I will stop at the engine half if so — I have the route half written and host-tested but no MOSS-Image checkpoint or host here, so it stayed out of this pull request. |
|
Thanks @devopsfortunato, for the clean rebase and for asking on #1573. Once @BobClawblaw has answered there, this can land after the cut. |
e5ca1ce to
d005848
Compare
|
Thanks @devopsfortunato. One small thing while #1573 settles: the rebase comment names |
88a873e to
127762e
Compare
|
Thanks @devopsfortunato for the rebase. The code is the same as what we read, every existing engine builds and |
127762e to
47e109d
Compare
47e109d to
d116edf
Compare
|
Thanks for the two notes, @ashhart - both are done, and the body now names the current head. The rebase broke the lean check, and that is now fixed. On The body now names head Checks on this head
Review: the round's diff went through GLM-5.3 - verdict approve, no correctness findings, and the check that the three compacted declarations kept their field names, order, types, defaults and docstrings. Not run, and why: no device or checkpoint receipt - nothing in the tree sets |
|
Thanks @devopsfortunato, and for catching the 607 lines. The three one-liners change no name, type or default, and the image API is as we read it. We'll set the CHANGELOG line's citation at landing. It goes in the landing queue once our own cells on this head are in: main against it, with tokens equal and speed level. Then #2253 can restack onto main. |
|
/atc exchange 6a4409a46106f033561894b44060d39bcd695fc0abc0e75e7bae6b98762dabf1 |
What this changes
34d42fcc8), one commit (d116edf3a).atc overlap --jsonon this head:complete: false(pullFilesUnavailable,timeLimit), 5 changed files. /v1/models: each model lists what it supports, from the same facts the refusals read #2157, CUDA: #1764's BF16 prompt tile for Qwen ModelOpt, Nemotron and Dory, plus two exact Qwen prompt kernels (#1762) #2008 and core: Gemma 4 and Qwen 3.5 on CUDA load with direct reads, 28-31 % faster cold; Flash Next shares the read-ahead #2282 each conflict in one file,CHANGELOG.md, where both sides add a bullet under the same## Unreleasedanchor. /v1/models: each model lists what it supports, from the same facts the refusals read #2157 also editszig/src/core/engine_api.zig, where the three-way merge is clean (no conflict markers), so its edits land beside this insertion. Nothing else overlaps.Before: the engine API can hand back tokens, audio and decisions, but a model that is not a token family has no way to hand back a picture. After: an engine may state an optional image capability beside
SpeechInfo—Info.image(max_side,max_pixels,max_steps,max_prompt_tokens) plus a defaulted, additiveVTable.submit_imagereached throughEngine.submitImage. The engine encodes one PNG a request and appends it to a caller-owned byte list using the allocator the caller puts in the request, so the caller owns and frees it afterfinished; an engine that does not set the entry is unchanged, andEngine.submitImageanswerserror.Unsupported.ImageRequestcarries the prompt and negative token ids,width,height,steps,guidance,seed, the caller'sImageResult, the caller'spnglist and itsalloc. Nothing in the tree setsInfo.imageyet, so no served behaviour changes. The follow-up pull request carries the route and the MOSS-Image host; no route, noserverecognition, no/v1/modelschange and no host are here.This round: the lean limit
After the rebase onto
34d42fcc8,zig/src/core/engine_api.zigstood at 607 lines, and the lean rule caps a source file at 600, so CI'slean checkstep failed on this branch. This round only fixes that: three existing declarations that were spread over four lines each (Stop,Admit,Explain) are now one-liners, the style this file already uses forMemoryandPromptCacheCapacity. No name, type, default or docstring changed, no code moved, and the file is 598 lines. The image API surface is untouched.Checks on this head
d116edf3a(rebased on34d42fcc8).python3 tools/zig/lean_check.py, on the base (34d42fcc8) and on this head: 325 problems each, andtools/zig/lean_new.pyreports 0 new findings..zig-version, an empty Zig cache):zig build test --summary all— exit 0,Build Summary: 569/569 steps succeeded; 7359/7420 tests passed (61 skipped), 4m15s wall.zig fmt --checkon the changed Zig files — exit 0;git diff --check— exit 0.zig/src/core/image_api_test.zig— the entry is reached with exactly the caller's fields; the appended PNG is freed with the caller's allocator understd.testing.allocator(a leak or mismatch fails); a vtable without the entry answerserror.Unsupported;error.Busy/error.InvalidRequestcome back unchanged;Info.imageis reported exactly andInfo{}has none; thefinishedevent reaches the caller's sink with the reason the engine set.Info.image,VTable.submit_imagenorEngine.submitImageexists, so the file cannot compile against main. That they run in CI is shown by a planted control: changing one expected width to 999 makeszig build testexit 1 namingcore.image_api_test.test.the image entry is reached with exactly the caller's fields(and the same test under the engine-seam step); restored, exit 0.zig/src/core/engine_api_test.zig, the engine seam's own test list (run bybuild.zig's api module on Linux and Windows and byzig/build/metal_targets.zigon macOS), notcore/root.zig. The core root test graph carries nolanesmodule, so a test importingengine_api.zigthere fails withno module named 'lanes' … referenced by engine_api.Engine.VTable; wiring it besideengine_apineeds no build change and runs on both branches.d116edf3a: green — all seven checks (zig,zig-windows,zig-macos-dev-tools,fmt,release-paths,benchmark-clientson ubuntu-latest and windows-latest).Info.image, so no request reaches a device and no picture exists to compare. macOS, Metal, HIP and XPU were not run; this change has no backend code.Receipt
n/a — this change cannot reach inference: it adds an API surface and its tests, and no engine implements it yet, so no token, logit or picture is produced. The follow-up host's pull request carries the receipt for a served picture.
Checklist