Repository navigation
Server: one engine flag for models whose replies carry no reasoning block - #1933
Conversation
|
Thanks @laudney, reviewed: this lands. |
c9aaa6d to
51e7a75
Compare
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name dots.mocr. The dots refusal test now asserts that the registered dots family declares no reasoning block, which keeps its OCR text in message content. Only the dots assertion is added, so later OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name Nemotron Parse. A new registry test asserts that the registered Parse family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
Some families, such as OCR models, never emit a reasoning block, yet share chat templates whose enable_thinking switch the server honours. Info gains a reasoning flag, default true; when false, the prompt still renders with the requested enable_thinking, but the reply is all answer content. chat.prepare computes the reply-side switch once and keeps it on Prepared.reasoning, which feature checks, the think budget and reserve, logprobs, the think-end marker, the reasoning split and the empty-content warning now read; the startup thinking line follows it too. A native HIP family opts out with a `reasoning` decl, and HTTP tests cover both settings in both response modes.
The registry test required every registered HIP family to keep reasoning on, so each OCR family that declares replies without reasoning would have had to edit the shared test. The two inline families already prove that the declaration turns it off and that its absence keeps the default, so the registry loop is dropped.
51e7a75 to
6c38647
Compare
|
Thank you @laudney. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes. |
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from #1933 checks the hook on two anonymous namespaces, so it does not name GLM-OCR. The GLM-OCR flag test now asserts that the registered family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name dots.mocr. The dots refusal test now asserts that the registered dots family declares no reasoning block, which keeps its OCR text in message content. Only the dots assertion is added, so later OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name Nemotron Parse. A new registry test asserts that the registered Parse family declares no reasoning block, which keeps its OCR text in message content. Only this family's assertion is added, so the other OCR families do not conflict with it.
|
Thanks @laudney! This is in TensorFold 1.2.2, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.2 |
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name dots.mocr. The dots refusal test now asserts that the registered dots family declares no reasoning block, which keeps its OCR text in message content. Only the dots assertion is added, so later OCR families do not conflict with it.
The registry test from ashhart#1933 checks the hook on two anonymous namespaces, so it does not name dots.mocr. The dots refusal test now asserts that the registered dots family declares no reasoning block, which keeps its OCR text in message content. Only the dots assertion is added, so later OCR families do not conflict with it.
The registry test from #1933 checks the hook on two anonymous namespaces, so it does not name dots.mocr. The dots refusal test now asserts that the registered dots family declares no reasoning block, which keeps its OCR text in message content. Only the dots assertion is added, so later OCR families do not conflict with it.
Part of #1927.
Some checkpoints never emit a reasoning block, but their chat templates still read
enable_thinking. The OCR models in #1927 are like this. Today, the server splits their replies as if they could think: with thinking on, which is the default, the whole OCR text lands inreasoning_content, andcontentstays empty. This PR adds one engine flag for such models. A family that sets it gets the template's own prompt rendering, and its reply is always answer content.reasoning_content,contentemptycontentenable_thinkingInfo.reasoningdefaults totrue)pub const reasoning = false;thinkingwith this capabilitychat.prepare(Prepared.reasoning)Changes
engine_api.Info.reasoning: bool = true.chat.preparekeepsthinkingfor prompt rendering. It computesreasoning = thinking and srv.info.reasoningonce and stores it onPrepared. Each reply-side use reads it:reasoning = false, the Harmony fallbacks (parseHarmonyinfinishandstreamingVisiblein the stream path) do not run.native/hip.zig: a family'spub const reasoning = false;sets the info flag. A family without that declaration keepstrue.Gates
hf_eos_http_test.zig, through the realHfTextand the OpenAI handler, both streamed and whole:reasoning = falseunder default, explicit-on and explicit-off thinking: the prompt follows the template, the reply is allcontent, andreasoning_tokensis 0.trueunder the same three settings: the split works exactly as before.<|channel|>analysis<|message|>plan<|channel|>final<|message|>OK) withreasoning = falsestays complete answer content. This test fails if the stream-path guard is removed.native/hip.zig: the registry test shows that a declared family getsfalseand every other family keepstrue.zig build test: 465/465 steps.zig build test -Dhip=true: 492/492 steps.test-cuda-hostand thenative-hipbuild pass.chat.zigstays at 600 lines.