[jayleaton]

What this changes

#1208 made a DeepSeek-V4.1 checkpoint without a Jinja template render through the native encoder (serve/template.zig). The public Flash-1017 EXL3 pack still carries September's chat_template.jinja for older loaders, so main renders every chat prompt through that stale file and not through the encoder that matches the release's encoding.py. Prompt ids, effort defaults and tool rules all differ.

• Precedence (DeepSeek-V4 and V4.1 model types only): the native encoder renders chat even when the checkpoint ships a Jinja template, unless --chat-template names an override. This is a deliberate DeepSeek-wide contract: a customized DeepSeek checkpoint's own Jinja is ignored unless the operator passes --chat-template. Other families keep checkpoint-template loading, and their tests are unchanged.
• The native path also covers what the Jinja path did:
  • Image parts become the image token when the server serves images, and are still refused otherwise.
  • A request's response_format schema appears as the encoder's ## Response Format block, as the release does. TF_DSV41_SCHEMA_PROMPT=0 turns that off.
  • /tokenize and the cache-prefix probe render the same way.
• A second commit stops the encoder from escaping its own image marker. Message text is escaped as it is read, so a typed <｜deepseek_image｜> is still escaped, and only the hook's marker renders as template text outside the request spans.

Issue: follows #1208, whose remaining check compared the encoder against the 1017 encoding.py; the goldens here are that check. Stacked on: nothing (based on main 3f0a715c4; the 51 later main commits, to 22f3d289a, touch none of its files).

Receipt

• Environment, with the TensorFold commit, MLX or PyTorch version, machine and GPU, checkpoint and revision: head f61e23911740 on main 3f0a715c4; Zig 0.17.0; host checks on a Linux aarch64 DGX Spark (GB10, CUDA 13.0). Real-pack boot on two GB10s, TP=2 over RoCE, checkpoint jayleaton/DeepSeek-V4.1-Flash-1017-EXL3-2.8bpw revision 859396591c34, TF_EXPERIMENTAL=deepseek.
• Exactness: prompt token ids now equal the release's encoding.py goldens through HfText, including the response_format cases. Known gap: the OpenAI message normalization joins a text-parts array before the encoder sees it, whereas encoding.py puts \n\n between parts. That predates this PR and is not changed here.
• Decode speed before and after: n/a.
• Prompt speed before and after: n/a; rendering only.
• Host tests: the committed encoding.py goldens through HfText; ignored checkpoint Jinja; the --chat-template override; served and refused images; json_object, string and object schemas; the knob; and a provenance test that tells a typed marker from the hook's marker. Updated upstream tests are the ones that pinned "checkpoint Jinja wins" for DeepSeek.
• Tests run, with pass, fail and skip counts:

  zig fmt --check build.zig zig                      exit 0
  git diff --check 3f0a715c4..HEAD                    exit 0
  python3 tools/zig/lean_check.py                     base 425 problems, head 425, new 0 (lean_new.py)
  zig build test-cuda-host --summary all              exit 0, 271/271 steps, 3908/3917 tests passed, 9 skipped (with test-server-cpu)
  zig build test-dsv41-shared test-dsv41-state test-dsv41-forward test-dsv41-kv test-dsv41-device-host test-dsv41-draft test-dsv41-cuda-host test-draft-dsv41-host test-server-deepseek-chat-host test-tool-visibility-http-host test-golden test-server-security  exit 0, 64/64 steps, 584/586 tests passed, 2 skipped
  

• Real-pack TP=2 boot of main + every PR in this series (and the DeepSeek image path that is not yet proposed), at 83ba53298924: passed. Both ranks reached readiness; the recipe smoke, the canary and a strict tool call passed; MissingTritonVariant 0 in both rank logs; 2048 greedy tokens in 51.1 s with token_sha256 7538b20557b1…, the same as the downstream build of this series.
• Not run, and why: other DeepSeek-V4 checkpoints with customized templates.

Checklist

• Every token still goes through the lane rounds. No serial path, and nothing that needs drafts off.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. (Not measured for this change alone.)
• No precision traded for speed. If the bits change, the description says which sums change and why.
• A new or changed GPU kernel is registered with the kernel contract (docs/kernel-contract.md); each backend's SUMMARY line is in the receipt. (No kernel changes.)
• Prompt processing is no slower than the last release. (Rendering only.)
• The description says which platforms I ran, Metal M1 to M5 and CUDA, and which I could not.
• New tests fail before the change, pass after it, and skip cleanly without their dependency. (No reverted-fix control was run for this PR.)
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @jayleaton, the goldens are the check #1208 left open. Since this changes every DeepSeek-V4.x prompt that used the checkpoint's Jinja, could you run the two unticked items on current main and post the counts?

1. The new goldens and the ignored-Jinja test against main's renderer, where they should fail.
2. Drafted equals "draft": false on the 1017 pack at TP=2.

Also: this ignores a customized DeepSeek checkpoint's own Jinja unless --chat-template is passed. Does the start-up log say which renderer it chose, so an operator can tell?

[jayleaton]

Thanks @ashhart. Here's the host part; the TP=2 part follows below.

1. The new tests against main's renderer. I put this PR's deepseek_chat_test.zig on current main 3ab49e300 with main's renderer unchanged. The only addition was inert stubs for the four names the tests reference: RenderOptions.response_format, RenderOptions.images, schemaPrompt and HfText.schema_prompt. Then I ran zig build test-server-deepseek-chat-host:

test	main 3ab49e300	PR head c5ba25634
goldens: missing Jinja uses the encoding.py goldens (234 cases, 56 with a response_format)	fail	pass
ignored Jinja: an explicit override wins, a checkpoint Jinja does not	fail	pass
ignored Jinja: tool-only Jinja metadata does not replace the native encoder	fail	pass
response_format shows its schema unless TF_DSV41_SCHEMA_PROMPT=0	fail	pass
whole DeepSeek chat: step	22 pass, 4 fail (26)	21/21 pass

The other 22 tests pass on both. The totals differ because the PR is based on 3f0a715c4, and main has since added five more DeepSeek chat: tests.

Start-up log. It didn't say. Nothing logged which renderer was chosen, so an operator couldn't tell that a customized checkpoint's Jinja was being skipped. I pushed c5ba25634, which adds one start-up line:

[tensorfold] chat template: the DeepSeek-V4.x encoder (encoding.py); the checkpoint's own chat template is not used, --chat-template FILE uses it

When the checkpoint ships no Jinja, the line is the DeepSeek-V4.x encoder (encoding.py). Other models get the --chat-template file, the checkpoint's, or none: chat requests are refused, completion prompts are served. The ignored-Jinja tests now check each of these. A DeepSeek checkpoint's Jinja is still read only to set that flag, and the encoder never renders it.

Gates at c5ba25634, x86-64 Linux host, no GPU:

• zig fmt --check: clean. git diff --check: clean.
• Lean checker: 425 problems on the base and 425 on the head, 0 new.
• test-server-deepseek-chat-host test-golden test-server-security test-pull-security: 46/46 pass.
• test-cuda-host test-server-cpu test-kernel-contract: 4045 pass, 9 skipped.
• zig build test: 2636 pass, 18 skipped.
• One unrelated failure: keepalive_test "a silent engine's stream gets keepalives before its first token" crashed in two runs while the dev box was loaded. It passed when the step ran alone. It's a socket-timing test this PR doesn't touch.

2. Drafted equals "draft": false at TP=2 on the 1017 pack: not run yet. I have no Spark window, so these numbers will follow when one opens. The run is prepared: drafted, plain, repeat and batched token ids for code and chat at C1 and C4, plus a response_format request and a tool turn, all through /v1/chat/completions on c5ba25634. It also records the new chat template: line.

[ashhart]

Thanks @jayleaton. That answers it: the four tests fail on main's renderer and pass here, and the new chat template: line tells an operator when a DeepSeek checkpoint's own Jinja goes unused and how to use it. Since you
have no two-GB10 window, we'll run drafted against "draft": false on the 1017 pack at TP=2 on ours, code and chat at
C1 and C4, with a response_format request and a tool turn, as you prepared. If those are equal, we'll land it.
@gilby, #1722 sets the same RenderOptions.images, so please take this PR's form when you rebase.

[gilby]

Thanks @ashhart, and @jayleaton. Agreed: #1722 takes this PR's form. When this lands, I'll rebase #1722 onto it and drop #1722's own image path in the renderer: the image_token option and the U+FDD0 mark in serve/template.zig, its line in hf_text.zig and its cases in deepseek_chat_test.zig. The Metal server will then get the image token from this PR's image hook (options.images → imageMarker), with a typed <｜deepseek_image｜> staying escaped text as your new test checks. RenderOptions.images and .images = srv.info.vision != null in renderIds are already word for word the same as here, comment included, so nothing in this PR needs to change for #1722.

[ashhart]

Hi @jayleaton, a quick status from our side: this one is waiting on us, not you. We still owe the TP=2 run we described
(drafted against "draft": false, code and chat at 1 and 4 streams, with a response_format request and a tool turn).
It's in our queue for the two-GB10 window. If main moves under it before then and you'd like us to do the rebase, say so
here and we will. @gilby, #1722 stays lined up behind it.