DeepSeek-V4.1-1017: /v1/chat/completions returns 500 "no chat template"; the server never calls the family encoder (serve/template.zig) #1208

What happens

The DeepSeek build of the native server returns HTTP 500 on every /v1/chat/completions request for a DeepSeek-V4.1-Flash-1017 checkpoint:

this checkpoint has no chat template: send a completion prompt instead

/v1/completions serves normally on the same boot. The 1017 release ships its prompt format as Python (encoding/encoding.py). It has no chat_template.jinja, and its tokenizer_config.json has no chat_template key. The server only renders chat through a Jinja template from the checkpoint. The family's own chat encoder, zig/src/families/deepseek_v41/serve/template.zig, is in the tree and its goldens pass, but nothing in zig/src/server calls it.

What we ran

• PR deepseek_v41: four-rank tensor parallel (whole-block shares, padded prefill heads) #1206's head 9e2f85bc on main 958efa3b, the DeepSeek build of zig/src/server/main.zig (tensorfold-dsv41 serve, followers tf-dsv41 follow), CUDA, NCCL, world 4 on four GB10s, --context 4096 --no-thinking.
• We added two local commits so our pack loads: fp16 router bias and DSpark confidence head, widened to fp32. Neither touches the server or the text path.
• Between 958efa3b, deepseek_v41: four-rank tensor parallel (whole-block shares, padded prefill heads) #1206's head and current main c9fab624, there is no diff in hf_text.zig, prompt.zig, model_text.zig, main.zig or zig/build/dsv41_draft.zig.
• Checkpoint: a 2.9 bpw EXL3 pack of deepseek-ai/DeepSeek-V4.1-Flash-1017. Its tokenizer.json, tokenizer_config.json and encoding/encoding.py are byte-identical (sha256) to the release's at 1bae0e10.

The failing request is tools/bench_concurrent.py's chat prompt, run as the #1206 description gives it:

{"model": "dsv41", "max_tokens": 256, "temperature": 0, "stream": true, "stream_options": {"include_usage": true},
 "ignore_eos": true, "seed": 1234,
 "messages": [{"role": "user", "content": "Explain how matrix multiplication uses a GPU in plain English, then give a small numerical example."}],
 "chat_template_kwargs": {"enable_thinking": false}}

Every chat cell failed: temperatures 0 and 1.0, at 1 and 4 streams. Rank 0's log has 44 "POST /v1/chat/completions HTTP/1.1" 500 lines and no chat 200 in any boot. The code cells (/v1/completions, 44 × 200) all passed, with alone and serial equal.

The code path (main c9fab624)

1. zig/src/server/main.zig:124 always loads the text side through loadText → hf_text.HfText.load (:162-163), whatever the family. Its only source of a template is --chat-template.
2. zig/src/server/hf_text.zig:46-50: with no --chat-template, readTemplates (:97-130) looks for chat_template.jinja, then chat_template.json, then tokenizer_config.json's chat_template. The 1017 release has none of these, so t.source stays empty, and :58-66 compiles nothing. t.compiled stays null and load succeeds.
3. zig/src/server/hf_text.zig:304-308: renderFn finds no compiled template and returns error.Template with the message above.
4. zig/src/server/prompt.zig:41-43: renderIds turns error.Template into cx.fail(.server, ...), which is 500 (errors.zig:49). docs/api.md:387 says a template refusal is a 400. A missing template isn't quite that case, but a 4xx would arguably fit better.
5. The family's encoder:
   • zig/src/families/deepseek_v41/serve/template.zig:1 is "DeepSeek-V4.1 chat encoding: OpenAI-shaped messages to prompt text". render is at :365, and serve/root.zig:3 exports it.
   • Its only callers are serve/golden_test.zig:110-187, which checks it against encoding.encode and the V4.1 release Jinja.
   • zig/build/dsv41_draft.zig:63-65 adds dsv41_serve to the server module ("the family's text side"), but no file under zig/src/server imports dsv41_serve.
6. zig/src/server/model_text.zig:1 says the tokenizer and template side is where "the tok and jinja branches plug in". Text.render (:113) is a vtable slot, and HfText is the only implementation (hf_text.zig:227).

Other families don't supply a template in a way DeepSeek misses. Qwen, GLM, Nemotron and the rest all get theirs from the checkpoint's Jinja through HfText. No family has a render hook today. DeepSeek is the one family whose release ships no Jinja, and the one family with a native encoder in the tree.

Two more things suggest the wiring is missing rather than never planned:

• The family declares itself chat-only: serve_engine.zig:23 sets .raw_text = false, and benchmark_model.zig:55-58 reads it.
• DeepSeek-V4.1: serving host, release tokenizer and the server's family hook #702 describes a server family render/reader hook for model_type: deepseek_v41 (srv.family / family.render in prompt.zig). Its closing comment says 1.1.1 includes those hooks. We can't find them on main or at v1.1.1 / v1.1.2: git log -S'srv.family' and git log -S'family.render' are empty.

Raw completion is not a substitute. #1019 shows a bare prompt on V4.1 often ends at once with EOS. On #1206 we saw the same thing: concurrent completion prompts with no assistant turn returned one token each.

Host reproduction

This runs without a GPU. We added a test to hf_text.zig (scratch only, not proposed as-is). It loads a directory laid out like the 1017 release:

• fixtures/serve/mini_tokenizer.json as tokenizer.json
• a tokenizer_config.json with only bos_token/eos_token
• an encoding/encoding.py

Rendering one user message then returns error.Template with exactly this checkpoint has no chat template: send a completion prompt instead. With the test added, zig build test-server-cpu gives 614/614 passed.

A second scratch test passes V4.1's release chat_template.jinja (deepseek-ai/DeepSeek-V4.1-Flash) as the override. It compiles and renders <｜begin▁of▁sentence｜><｜User｜>hi<｜Assistant｜></think> with thinking off. So --chat-template <V4.1's jinja> looks like a workaround on current main. #847 has a Metal run that served 1017 with V4.1's Jinja. Another comment there reports the template compiler rejecting DeepSeek's base template (*args and **kwargs calls are not supported); current main compiled the V4.1 file in our host test. We did not run it on the GPU (see below).

What a fix would touch

Option A, wire the native encoder (our preference, since it is the reference-checked one):

• Let an engine module (or native_engines) provide a model_text.Text render for its model_type, used when the checkpoint has no template and --chat-template is not given.
• The DeepSeek implementation would map RenderOptions (tools, enable_thinking, reasoning_effort, add_generation_prompt) onto template.Options and template.render.
• Template-written spans need marking for provenance. Today template.render returns text only, while HfText returns Rendered with spans. That matters for Chat prompts: control-token spellings in request text (user, system, tool results) become control tokens #1117's control-token rule.
• Files: main.zig (choose the text), hf_text.zig or a small family text adapter, serve/template.zig (spans), and a server test like the one above.
• Tool-call parsing (DSML, serve/tools.zig) is a separate question. We have not looked at whether the server's existing tool parser reads DeepSeek's output.

Option B, document or ship a Jinja: the DeepSeek recipe tells users to pass V4.1's chat_template.jinja with --chat-template for 1017 checkpoints, or tensorfold pull places one. That needs no code, but chat would then rest on a template the family's goldens only partly cover.

The gate question: is wiring the family's native encoder into the server acceptable, and under what gate (the template goldens plus a server test)? Or do you prefer DeepSeek packs to carry a Jinja template and the server to stay Jinja-only? We'd be glad to do either once you pick.

Not tested

• --chat-template with V4.1's Jinja on the GPU at any world size. Tool calls, reasoning_effort and thinking-on chat through it.
• Whether V4.1's Jinja output equals template.zig's (or encoding.encode's) for 1017 beyond the one message above.
• World 1 or 2, and Metal. The text path does not depend on world size, but we only ran world 4.
• Current main c9fab624 on hardware. We ran deepseek_v41: four-rank tensor parallel (whole-block shares, padded prefill heads) #1206's head; the files above are unchanged between them.
• The raw HTTP response body. We have the server's log line (500) and the message from the code path and the host test, not a saved response body.