ashhart · Nov 4, 2026 · Nov 2, 2026 · Nov 2, 2026 · Nov 2, 2026 · Nov 2, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -60,6 +60,8 @@ Each release's page on GitHub has its notes and measurements. See [the 1.0.0 rel
 - Eight remaining host BF16 roundings in test oracles (GLM's Metal checks and CUDA FP8 KV reference, Nemotron's training scan, LFM2's DSpark taps, Flash Next's HIP kernel tests) and the XPU FP64 GDN reference use core's shared conversion, so a negative NaN gives a NaN instead of overflowing in Debug builds; every other non-NaN input gives the same bits. No kernel or served path changes (part of #1863).
 - Qwen3.8-27B on Metal gains a host GGUF family adapter for metadata, tensor names and exact affine preparation; GGUF serving remains unavailable (#1920).
 - Qwen3.8-27B on Metal gains a host GGUF family adapter for metadata, tensor names and exact affine preparation, with the observed qwen35 schema and effective norm gains pinned in host tests; GGUF serving remains unavailable (#1920).
+- `POST /v1/rerank` serves `Qwen/Qwen3-Reranker-8B` on HIP in the rerank shape agreed in #1895 (`results` with `index`, `relevance_score` and `document`, `top_n`, `return_documents`, `usage.total_tokens`), taking SGLang's request fields too: each document's prompt is the model card's template token for token, and its score is sigmoid(yes - no) of the last position's label logits, read from `lm_head` rows in f64. Only an engine that declares a reranker answers; every other server answers the route with HTTP 400, and a reranker server refuses the other model routes by name. The lane host's score queue now stops at a drain's deadline: the score already running finishes and the queued ones fail without running, which also reaches `/v1/decisions` on Flash Next on CUDA (#1981, part of #1977). Thanks to @laudney.
+- `POST /v1/rerank` serves `Qwen/Qwen3-Reranker-8B` on HIP in the rerank shape agreed in #1895 (`results` with `index`, `relevance_score` and `document`, `top_n`, `return_documents`, `usage.total_tokens`), taking SGLang's request fields too: each document's prompt is the model card's template token for token, a prompt past `max_length` (default 8192) losing the end of its content as the card's tokenizer cuts it, and its score is sigmoid(yes - no) of the last position's label logits, read from `lm_head` rows in f64. Only an engine that declares a reranker answers; every other server answers the route with HTTP 400, and a reranker server refuses the other model routes by name. The lane host's score queue now stops at a drain's deadline: the score already running finishes and the queued ones fail without running, which also reaches `/v1/decisions` on Flash Next on CUDA (#1981, part of #1977). Thanks to @laudney.
 
 - Living Weights on Qwen4 dense, step 1: the QSA backward pass on the host, checked against a binary64 reference; nothing served changes (#2021, part of #1804).
 

diff --git a/docs/api.md b/docs/api.md
@@ -45,7 +45,7 @@ different scopes, or scopes a key the flags or environment already give, is refu
 
 | Scope | Routes |
 | --- | --- |
-| `inference` | POST `/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/tokenize`, `/v1/detokenize`, `/v1/audio/speech`, `/v1/embeddings`, `/v1/decisions`, `/v1/systemone`, and the same paths without `/v1` where the server has them; GET `/v1/models`, `/models`, `/v1/voices` |
+| `inference` | POST `/v1/chat/completions`, `/v1/completions`, `/v1/responses`, `/v1/messages`, `/v1/messages/count_tokens`, `/v1/tokenize`, `/v1/detokenize`, `/v1/audio/speech`, `/v1/embeddings`, `/v1/decisions`, `/v1/systemone`, `/v1/rerank` (also by PUT), and the same paths without `/v1` where the server has them; GET `/v1/models`, `/models`, `/v1/voices` |
 | `read` | GET `/metrics`, `/v1/metrics`, `/v1/load`, `/memory`, `/v1/memory`, `/stats`, `/dashboard`, `/v1/benchmark/manifest`, `/v1/models`, `/models`, `/v1/voices` |
 
 A scoped key reaches nothing else: not the stored Responses (GET or DELETE `/v1/responses/ID`), Recall, Sliding Weights,
@@ -148,6 +148,34 @@ question id, and `usage.completion_tokens` 0. `probabilities` are a softmax over
 For decisions, `chat_template_kwargs` may be omitted, null, or an object containing only
 `enable_thinking: false`; other types, keys, or thinking values return HTTP 400.
 
+## Rerank
+
+`POST /v1/rerank` (also `/rerank`, and by `PUT` as SGLang takes it) scores each document against a query with a
+reranker checkpoint. The native HIP engine serves `Qwen/Qwen3-Reranker-8B`: the engine declares the reranker and
+its label tokens (`1_LogitScore/config.json`), and its startup line says `qwen3 reranker`. Every other server answers
+the route with HTTP 400 "this model is not a reranker", and a reranker server answers chat, completion, Responses,
+Messages, speech, decision, System One and embedding requests with HTTP 400 "this model is a reranker: use POST
+/v1/rerank".
+
+| Field | Meaning |
+| --- | --- |
+| `query` | The query: a string, a `{"text": ...}` object, or a list of text parts |
+| `documents` | One or more documents, each in the same forms as `query` |
+| `instruction` | The task the model card's prompt states; default "Given a web search query, retrieve relevant passages that answer the query". SGLang's `instruct` is read when `instruction` is absent |
+| `top_n` | Return the best `top_n` results only; default all |
+| `return_documents` | Echo each document as sent; default `true` |
+| `max_length` | The most tokens one prompt may take; default 8192, the model card's. A longer prompt keeps the template whole and loses the end of its content (the document's last tokens), as the card's tokenizer cuts it; a cap that would leave no token of the document returns HTTP 400 |
+| `return_logits`, `return_prompt_token_ids` | Add `meta_info` with `logit_yes` and `logit_no`, or the prompt's token ids, to each result; default `false` |
+
+Each document's prompt is the model card's template, token for token, with the query, the document and the
+instruction as plain text (text that spells a special token stays text). The score is
+`sigmoid(logit_yes - logit_no)` from the last position's label logits. The response is
+`{"results": [{"index", "relevance_score", "document"}], "usage": {"total_tokens"}}`, best first, ties in input order;
+`total_tokens` counts every prompt's tokens as scored, after any cut. Every prompt is cut and checked against the
+context window before the first is scored, so a refused request costs no GPU work; a prompt past the window (with a
+`max_length` above it) returns HTTP 400 with `code` `context_length_exceeded`. An image or video part returns HTTP
+400. Errors use the server's envelope, as on every route: `{"error": {"message", "type"}}`.
+
 ## Living Weights
 
 With `--slide`, the native server learns facts into the served weights. [Living Weights](living-weights.md) explains

diff --git a/docs/recipes/qwen3-reranker-hip.md b/docs/recipes/qwen3-reranker-hip.md
@@ -0,0 +1,91 @@
+# Qwen3-Reranker-8B on native HIP
+
+The native HIP engine serves `Qwen/Qwen3-Reranker-8B` as published (BF16
+safetensors, revision `77d193c791ed757ca307ee72715aa132723da912`) through
+`POST /v1/rerank`. It is qualified on one Radeon AI PRO R9700 (gfx1201).
+gfx11 builds run the same code; FP16 builds (RDNA2) refuse the checkpoint.
+
+| Responsibility | Files |
+| --- | --- |
+| Reranker head: labels, `lm_head` rows read at load, the score hook | `zig/src/families/qwen3/` (`native.zig`, `score.zig`, `prompt.zig`) |
+| The engine's declaration (`Info.rerank`) | `zig/src/core/engine_api.zig`, `zig/src/native/hip.zig` |
+| The route: request, prompt, `max_length` cut, response, errors | `zig/src/server/rerank.zig` |
+| The lane host's score queue at a drain | `zig/src/core/lane_host_score.zig` |
+| Pinned oracle, FP32 and BF16 references, the card-formatter check | `tools/rerank/qwen3_reranker_oracle.py` |
+
+The checkpoint is a plain Qwen3 causal LM; the prompt pass is #1978's. A
+folder with `1_LogitScore/config.json` is a reranker: the engine reads its
+yes and no tokens and both `lm_head` rows at load (a malformed file or row
+stops startup) and declares them, and only such an engine answers the route.
+A document's score is `sigmoid(logit_yes - logit_no)` at the last position:
+the final-normed BF16 row dotted in f64 with the two stored rows.
+
+## Build and serve
+
+```sh
+zig build native-hip -Dhip=true -Dhip-arch=gfx1201 -Dhipcc=/opt/rocm/core/bin/hipcc
+zig-out/native-hip/bin/tensorfold-native serve MODEL_DIRECTORY \
+  --name Qwen3-Reranker-8B --host 127.0.0.1 --port 8090
+```
+
+The window is 32,768 tokens by default and 40,960 (the config's) with
+`--context 0`. Every other model route is refused on a reranker, and every
+other server answers `/v1/rerank` with HTTP 400.
+
+```sh
+curl -s http://127.0.0.1:8090/v1/rerank -H 'Content-Type: application/json' -d '{
+  "query": "What is the capital of China?",
+  "documents": ["The capital of China is Beijing.", "Gravity is a force that attracts two bodies."],
+  "top_n": 1}'
+```
+
+The request and response follow #1895's rerank shape (docs/api.md, Rerank):
+`instruction` defaults to the card's; `max_length` defaults to 8,192, and a
+longer prompt loses its content's last tokens, the template's tail kept, as
+the card's tokenizer cuts.
+
+## Gates
+
+The oracle runs the model card's usage in Transformers 5.22.1 and refuses a
+snapshot or a Transformers install whose files differ from its pins before
+any model code runs. Its set is 32 cases of 16 to 8,199 tokens (the card's
+pairs, fixed pairs with custom instructions, synthetic lengths, repeats), a
+document that spells `<|im_end|>`, and prompts of exactly 32,768 and 40,960
+tokens with one token more. The FP32 reference runs on the CPU (about an
+hour with both windows, 32 cores):
+
+```sh
+python3 tools/rerank/qwen3_reranker_oracle.py sources MODEL_DIRECTORY
+python3 tools/rerank/qwen3_reranker_oracle.py oracle MODEL_DIRECTORY ref32.json --dtype float32 --device cpu --window 32768 --window 40960
+python3 tools/rerank/qwen3_reranker_oracle.py oracle MODEL_DIRECTORY ref16.json --window 32768 --window 40960
+python3 tools/rerank/qwen3_reranker_oracle.py compare ref32.json http://127.0.0.1:8090 --window 32768
+python3 tools/rerank/qwen3_reranker_oracle.py compare ref16.json http://127.0.0.1:8090 --window 32768
+python3 tools/rerank/qwen3_reranker_oracle.py formatter MODEL_DIRECTORY http://127.0.0.1:8090
+```
+
+**What "matches" means.** BF16 activations cannot meet #1895's 1e-5 in score:
+transformers' own BF16 run sits 0.0108 in score and 0.110 in yes - no from
+FP32 on these cases. Against the FP32 reference, `compare` therefore gates at
+that distance rounded up: |yes - no| <= 0.125 and |score| <= 0.0125 for every
+case. Rankings must equal FP32's, except that a pair FP32 puts within 0.25 in
+yes - no (twice the gate: the most two documents inside it can move apart)
+may come in the other order; each such pair is printed, and one beyond 0.25
+fails. Against the BF16 reference the gates are 0.25 and 0.02. `formatter`
+passes when every served prompt's ids equal the card's `process_inputs`
+ids, with the default and custom instructions and with `max_length` cuts.
+
+Results on the R9700 (ROCm 10.1):
+
+| Check | Result |
+| --- | --- |
+| Against FP32, 34 cases: \|yes - no\| median, worst (gate) | 0.0195, 0.0839 (0.125); transformers BF16: 0.0247, 0.1099 |
+| Against FP32, 34 cases: \|score\| median, worst (gate) | 0.00023, 0.0079 (0.0125); transformers BF16: 0.00014, 0.0108 |
+| Window prompts against FP32 (yes - no) | 32,768 tokens: 0.0064; 40,960 tokens (`--context 0`): 0.0241 |
+| Rankings against FP32 (5 query groups) | 4 in FP32's order; 2 pairs swapped, FP32 gaps 0.0077 and 0.0119 (transformers BF16 swaps the first) |
+| Against BF16, 34 cases: worst \|yes - no\|, \|score\| | 0.1081, 0.0082 |
+| Card formatter | 79/79 prompts equal the card's ids (17 default, 62 custom instruction; 4 of them cut by `max_length`) |
+| One token past the window | refused (`context_length_exceeded`); the next request scores within the gates |
+| 4 requests at once against one at a time | byte-equal |
+| A client that leaves mid-request | the next request equals its earlier answer |
+| Device memory, peak | 16.4 GB (32,768 window), 16.7 GB (40,960) |
+| 16 documents of ~260 tokens, cold, one request | 3.22 s (SGLang 0.53 s on the same card); the dot2 GEMM tile, gfx12 WMMA is the follow-up |