--vision-image-tokens is capped at 16,384 again: d5e69ee4's raise to 262,144 didn't survive #940's merge #1198

Posted by Claude (Anthropic's AI assistant) on behalf of @sethforprivacy, as part of our local-inference work on a Mac Studio M5 Ultra.

What we found. d5e69ee4 (@MiaAI-Lab, "server: --vision-image-tokens up to 262144") arrived on #940's branch. The merge of #940 (6e4a0b64) kept main's line, so main and every release since still refuse anything above 16,384:

• 6e4a0b64^2 (nemotron omni: video input through the RADIO tower #940's head): expected 16 to 262144
• 6e4a0b64, and zig/src/server/cli.zig:348 on v1.1.0, v1.1.1, v1.1.2 and main today: expected 16 to 16384

The merge message doesn't mention it. The #940 review was about the Super VL and Bolt row pools, so we couldn't tell whether keeping 16,384 was intended.

Why it matters to us. We serve Qwen3.8 Flash Next on Metal with --parallel 16 and --vision-max-images 32. Before 1.1.x we served --vision-image-tokens 65536, and 1.1.2 refuses that at startup, so we now serve 16,384.

A request's images split the budget evenly (server/images.zig:169, image_tokens / n_images), and each image is resized to its share. So:

• At 16,384, a full-size 2560x1440 screenshot (about 3,600 tokens) is shrunk from the fifth image on, along with every image before it.
• At the native default (4,096, 4 images), a second screenshot already shrinks the first.

Shrinking an earlier image changes its tokens, so a conversation that adds a screenshot can't resume its kept prefix. We'll come back to that in a PR on Flash Next image resume that we're preparing.

On Flash Next the cost of a higher budget is already counted: flashnext_images.plan charges the request's rows buffer ([tokens, 4, 2560] bf16, about 20 KiB a token) against the load ceiling at the configured budget.

Question. Was keeping 16,384 intended? If not, would you take d5e69ee4 again, or a per-family cap if 262,144 is too much for Super VL's or Bolt's pools?