[BobClawblaw]

What this changes

Adds the ricursive family on the native Metal engine: ricursiveAI's Ripple-65B-Preview served from the publisher's own
releases as published, read in place from their shards, nothing converted. Before, ricursive checkpoints were
refused as having no backend.

• ricursiveAI/Ripple-65B-Preview-NVFP4 at
  99eea4477a001a41b86914e41feb4b2ed86105e5 (53 GiB): ModelOpt NVFP4 MLPs; attention, embeddings and head bf16.
• ricursiveAI/Ripple-65B-Preview at
  240d5966cbe00f0627c699e59ef58560b60303cf (121 GiB): the bf16 release.

The full account is docs/recipes/ripple-65b-metal.md.

• Issue: Fixes Ripple-65B (ricursive) on native Metal: the publisher's BF16 release #2171 (it named the bf16 release; the publisher's NVFP4 release is added because bf16 leaves little room
  for caches on one Mac).
• Stacked on: main (c70d67bb2), six commits: H3's contract registration moved into its own Suite (so
  tests/contract/metal_main.zig stays at 600 lines or fewer, now 597 beside main's masked_values suite), the family,
  the context cap and its notice, the doc rows from review, the contract at o_proj's reduction, and the CHANGELOG line.
• atc overlap --json: no conflict with main. Shared hub files overlap with other open PRs (contract registry,
  native/metal.zig, tests/contract/metal_main.zig); Implement GLP control vectors with a fused Metal kernel for Qwen3.8-27B #1835, kimi: K3 and K3.1 through the native server on Metal #2221 and kimi: experimental K3.1 index operators on Metal, native serving stays capped #2228 would conflict there, whichever lands
  second rebases. No conflict resolved in this branch.

Things to say up front

• Activations stay bf16. The publisher serves the NVFP4 release with SGLang's flashinfer_cutlass, which also
  quantizes activations to fp4 (W4A4). This engine runs the release's NVFP4 weights exactly (E2M1 × unsigned E4M3 per
  16 × fp32 global, the global applied to the fp32 sum before the one bf16 rounding) with bf16 activations and fp32
  sums, so the numbers here are that model, not the publisher's W4A4 kernels.
• What NVFP4 costs (teacher-forced, 2,586 positions): the engine on NVFP4 against the bf16 release's float64 pass,
  mean KL 0.0060, top-1 96.1%; the NVFP4 checkpoint's own float64 pass against the same, 0.0059 and 95.8%. The cost is
  the publisher's quantization; the engine's own rounding is 0.0007 (prompt) / 0.0011 (decode) against the NVFP4
  float64 pass, and 0.00086 / 0.00073 for bf16. Against the release's own modeling_ricursive.py in bf16 (the
  reference's --release mode), the bf16 engine's mean KL is 0.00073 (prompt, top-1 98.7%) / 0.00069 (decode,
  98.8%), against 0.00076 for the float64 pass.
• One flip: at one position of a 5,200-token NVFP4 prompt, the prompt path's top token differs from decode and from
  float64 (float64 000 at 0.645, decode 000 at 0.621, the prompt path 096 at 0.855). It traces to the shared bf16
  tensor-unit GEMM on the attention projections, which matches MLX bit for bit and passes the contract at these shapes;
  bf16 does not flip there. Details in the doc.
• Caches are exact bf16. 64 of 80 layers slide over 1,024 keys in a 3,072-slot ring (window + 2,048-row chunk);
  every fifth layer is global (NoPE). 16.75 GiB a stream at 262,144 tokens.
• Context. --context defaults to and stops at 262,144, the longest checked (needles to 258,039 tokens); longer
  values, the card's 1M among them, are refused by name until a run at that length qualifies them.
• Thinking. Ripple decides for itself whether to think, and its <think>, </think> and <tool_call> are spelled
  from several BPE tokens, not single tokens. So a thinking_budget and logprobs are refused by name for this family,
  and reasoning_tokens reads 0.

Shared code

All outside the family folders, each inert unless the model is ricursive:

• engine_api.Info.reasoning_optional (default false; only Ripple sets it) → reply_text.Markers.optional: a reply
  that does not open a think block is all answer. server/chat.zig reads replies for a think block whenever that flag
  is set, whatever enable_thinking renders (the template has no switch). With the flag false, both expressions reduce
  to the previous ones; test-golden is unchanged (322/335 equal, 0 unexpected).
• native/metal.zig: the family entry, the open dispatch and the model name in two refusal strings.
• tests/contract/metal_llama*.zig, metal_limits.zig, registry.zig: llama's row projections, merge, argmax and
  GEMM retargeted at Ripple's shapes through new optional fields; llama's own cases keep their defaults.
• sources.zig, metal_targets.zig, cli/hub.zig, tensorfold.zig, server/server.zig: registration lines.
• THIRD_PARTY_NOTICES.md: the family follows modeling_ricursive.py and the release's SGLang model (Apache-2.0).

Checks on this head

• Head commit: c244847b6 (on main c70d67bb2; 70c1867b5 rebased, with CHANGELOG.md the only conflict and the other five commits' patches unchanged by git range-diff). ricursive: the publisher's DFlash2 drafter for Ripple-65B (stacked on #2231) #2260's head 8843987e8 sits on this one.
• Lean checker: base 155 problems, head 155, 0 new.
• Tests run on this head and on ricursive: the publisher's DFlash2 drafter for Ripple-65B (stacked on #2231) #2260's (each exit 0): zig fmt --check build.zig zig, zig build test --test-timeout 10m,
  test-kernel-contract, test-golden (322/335 and 55/55 equal to server goldens, 0 unexpected differences),
  test-server-security test-pull-security (24 passed, 0 failed, 0 skipped), check-release-paths, fuzz-cortheon
  and dev-tools. tf-kernel-contract --only ricursive on ricursive: the publisher's DFlash2 drafter for Ripple-65B (stacked on #2231) #2260's head: 75 kernels pass gates 1-4, 25 controls red as planted, 0 limit problems.
• Served check after the rebase over Server: tool calls completed just before the thinking close are returned when no answer follows (#2310) #2313 (tool calls before the think close): the recipe harness's structured plan
  on the NVFP4 release, served from 630f1fc4f (this head's Ripple code, before main's last 34 commits, which touch
  neither the server nor Metal): structured 18/18 obey, drafted == plain 12/12, a bad regex 400; tools 5/5.
• CI: all seven checks pass on this head (zig, zig-windows, zig-macos-dev-tools, fmt, release-paths, benchmark-clients on Ubuntu and Windows).
• Not run, and why: no quality suite. M1 to M4 are refused at load (prompts run on M5 tensor units); other RAM sizes
  untested.

Receipt

• Environment: Mac Studio M5 Ultra, 256 GB, macOS 27.2, Zig 0.17.0, Apple metal 32023.922; checkpoints and revisions
  above.
• Paths and platforms: Metal only (prompt chunks on M5 tensor units, decode windows on the row kernels), both releases.
  CUDA, HIP and pre-M5 Macs are not reached.
• Exactness: prompt chunks of 2,048, 1,000 and 333 rows give bit-identical logits; decode windows of 16 and 7 rows equal
  one-row steps bit for bit (both releases). Serving, through the recipe harness's quick plan on e2ca8c387: seeded
  sampling 9/9 equal on repeat and undrafted; drafted == plain 12/12; 12 concurrent replies and a 60-turn soak each equal
  to the request alone; a resumed turn equal to the cold one after a restart. Structured 18/18 and tools 5/5 rerun on
  68e1993d9. --context 262145 is refused by name.
• Decode speed (new family, no before; NVFP4, --context 262144 --parallel 2, greedy 256-token replies, recipe harness
  rather than bench_openai.py): 1 stream 10.1-10.2 tok/s, 99 ms a round, TTFT 0.61-0.74 s; 2 streams 18.6-19.1 tok/s
  total, 107 ms a round.
• Prompt speed (cold, fresh server, recipe harness prefill plan): 2,220 tokens 569 tok/s (3.9 s), 8,776 605 (14.5 s),
  35,015 587 (59.6 s), 139,723 434 (322 s). A resumed third turn (4,513 of 4,597 cached) starts in 0.67 s against 8.2 s
  cold. A passphrase a third of the way into 120,041- and 258,039-token prompts comes back exactly (273 s and 790 s).
• bf16 release, the same harness plans: 1 stream 4.9-5.8 tok/s, 195 ms a round; 2 streams 10.5-11.8 tok/s total, 180 ms
  a round. Cold prompts with thinking requested: 2,206 tokens 519 tok/s (4.3 s), 8,830 596 (14.8 s), 34,931 586
  (59.6 s), 139,870 459 (305 s); without thinking requested Ripple opened a think block itself on all four, so the
  plan's 8-token replies had no answer token to time. A resumed third turn starts in 0.59 s against 7.5 s cold.
• Licence: Apache-2.0, both releases at the revisions above. No drafter.

Kernel contract

• Registered: the family's glue (embed, add+norm, q/k norm+RoPE+cache write, gate, clamped SwiGLU), decode and prompt
  attention for sliding and global layers, the NVFP4 row kernels (1-32 rows), dequantization and GEMM, plus llama's row
  projections, merge, argmax and GEMM at Ripple's shapes, every reduction it runs (k 5,120, 10,240 for the attention
  output, and 40,960) (zig/tests/contract/metal_ricursive*.zig). None replaces an existing kernel.
• Fifteen planted controls, each editing one kernel's source; each turns its gate red.
• tf-kernel-contract --only ricursive, M5 Ultra, exit 0, with the k 10,240 cases (the run quoted in the doc):

SUMMARY 46 kernels: 46 pass gates 1-4, 0 fail, 0 known findings; 15 controls: 15 red as planted, 0 missed; 0 limit problems; gates 645430 ms, limits 140 ms

• Chips not run: M1 to M4 (refused at load), other M5 sizes.

[ashhart]

Thanks @BobClawblaw. This is a thorough first PR for a new family: bit-identical chunks and windows, 15 planted controls all red, and the NVFP4 cost measured against both float64 passes, which shows it's the publisher's quantization rather than ours. The family itself still needs a yes on #2171, so this is a review of the PR against its plan, not an acceptance. Three things:

1. --context admits up to 1,048,576, but nothing past 258,039 has run. Please take your offer and cap it at 262,144, refusing anything longer by name, until a run at the longer length qualifies it.
2. The plan's reference was the release's own modeling code. Your table compares against your float64 pass. Could you add the --release row: engine bf16 against modeling_ricursive.py in bf16, teacher-forced on the same 2,586 positions, with mean KL and top-1?
3. The bf16 release is the checkpoint Ripple-65B (ricursive) on native Metal: the publisher's BF16 release #2171 proposed. Please post one decode and prompt row for it through the harness, as you did for NVFP4.

One question as well: glue.metal follows the release's rounding points. Was any of the family translated from modeling_ricursive.py, rather than written from the config and weights? If so, it needs a THIRD_PARTY_NOTICES entry for the Apache-2.0 release code. And adding the NVFP4 release goes beyond the plan, so we'll confirm it on #2171 along with the family.

[BobClawblaw]

Thanks, Ash. All three are done, and the question is answered below. The branch is rebased on main 4a20028aa; the
head is 3a5954f38, three commits on the one from your review (the third registers llama's projections at
o_proj's reduction, k 10,240, with the contract: 46 kernels pass gates 1-4, 15 controls red). The rebase met #2221's Kimi line in
server/chat.zig; the merged expression keeps it as is and reduces to main's when markers.optional is false.

1. Context capped. --context now stops at 262,144 (config.max_context), and a longer value is refused by name
before the memory budget is consulted:

--context 262145: the native Ripple engine is checked to 262144 tokens; longer contexts are refused until a run at that length qualifies them

That is serve on the NVFP4 release, built from the cap commit before the rebase (only a comment changed since). The doc now says the card's 1M is refused until a run qualifies it.

2. The --release row. tools/zig/ricursive_reference.py --release runs the checkpoint's own
modeling_ricursive.py decoder layers and final norm in bf16 (torch on the CPU, a layer's weights at a time, the
release's own masks: is_causal on global layers, its sliding mask on the others). I teacher-forced it on the same
2,586 positions. The engine logits, regenerated on the cap commit's build, are bit-identical to the ones in the table.
KL(release ‖ engine):

engine bf16	against	mean KL	top-1
2,048-row prompt chunks	modeling_ricursive.py bf16	0.00073	98.7% (2553/2586)
16-row decode windows	modeling_ricursive.py bf16	0.00069	98.8% (2555/2586)
(the float64 pass, no engine)	modeling_ricursive.py bf16	0.00076	98.3% (2541/2586)

The engine is as close to the release's own bf16 forward as the float64 pass is: what remains is bf16 rounding order
on both sides. Rows added to the doc's table.

3. bf16 through the harness (the same build; --context 262144 --parallel 2, greedy 256-token replies):

streams	prose	code	structured	list	first token	a round
1	5.8	5.0	4.9	5.0	0.51-1.30 s	194.7 ms
2	11.5	11.0	10.5	11.8	0.86-1.81 s	179.6 ms

Cold prompts on a fresh server: 2,206 tokens 519 tok/s (4.3 s), 8,830 596 (14.8 s), 34,931 586 (59.6 s), 139,870
459 (305 s). Those are from the thinking-requested half of the plan. In the other half, Ripple opened a think block of
its own on all four prompts, so the harness's 8-token replies held no answer token to time, and that half recorded no
row. A resumed third turn (4,513 of 4,597 kept) starts in 0.59 s against 7.5 s cold, with the same reply as after a
restart.

The question. Yes. The family was written from the config and weights, but it follows modeling_ricursive.py
(and the release's SGLang inference/server/model.py) for its layer order and rounding points: the fp32 embed scale,
the fp32 gate sigmoid, each add rounded to the residual, where the value-residual lerp rounds. The reference's
float64 pass follows the same file. So THIRD_PARTY_NOTICES now has an entry under "Model release and library code
these files follow" for zig/src/families/ricursive/, glue.metal and tools/zig/ricursive_reference.py:
ricursiveAI, Apache-2.0, at release revision 240d5966. --release imports the release's own file from the
checkpoint at run time; nothing of it is copied in.

Understood on #2171: this waits for your yes on the family and on adding the NVFP4 release.

Checks, each exit 0 (the full set on the first two commits, 68e1993d9; fmt, test, test-golden and test-server-security
again with the contract commit on 610c67bf7; fmt and the native build on this head): zig build test --test-timeout 10m, test-golden (322/335 and 55/55 equal to
server goldens, 0 unexpected), test-kernel-contract, fuzz-cortheon, test-server-security test-pull-security,
check-release-paths, dev-tools -Doptimize=Debug, zig fmt --check build.zig zig. Lean: base 425, head 425, 0 new. Served on 68e1993d9: structured 18/18
(drafted == plain 12/12), tools 5/5 through the harness; a reply that opens its own think block is filed as reasoning,
one that does not as the answer, no tags in either.

[ashhart]

Thanks @BobClawblaw. The context cap refuses by name, the engine is as close to the release's own bf16 forward as the float64 pass is, and the bf16 speed row is in, so all three gaps are closed. Thanks also for the notices entry. The family itself, and adding the NVFP4 release, still need a decision on #2171, and we'll answer there.

[BobClawblaw]

Rebased onto main 0d7152955; the head is now 09dd4a491. On today's main tests/contract/metal_main.zig was at 600
lines, and the family's registration took it over, so the first commit moves H3's registration into its own Suite, as
5fe04fa did for the qwen27 attention rows. The file is at 597 lines. The family's commits are unchanged apart from the
rebase. The checks are in #2260's reply, run on the whole stack.

[ashhart]

Thanks @BobClawblaw for the rebase and for moving H3 into its own Suite. This still waits on #2171. For its next rebase: today's main adds a masked_values suite beside the H3 block in tests/contract/metal_main.zig, and on today's main server/reply_text.zig comes to 604 lines, so it needs to come to 600 or fewer. It will also need a CHANGELOG line under Unreleased.

[BobClawblaw]

Thanks @ashhart. Rebased on today's main (c3f64ca8a); head 70c1867b5, six commits:

• masked_values: tests/contract/metal_main.zig registers main's masked_values Suite, then the ricursive set, then the H3 Suite. It comes to 597 lines.
• server/reply_text.zig: now 600 lines. The optional-think test is a case table now, and it drops two cases whose paths other tests already cover (a partial <think> opener held while streaming, which the MiniMax-M3 test checks, and a finished tool call, which takes the same path as the bare reply). No code changed.
• CHANGELOG: changelog: #2231 under Unreleased adds one line under Unreleased.

CI steps exit 0 on this head: fmt, lean (0 new), zig build test, test-kernel-contract, test-golden. On #2260's head, those plus server and pull security, fuzz-cortheon and build options.

Since #2313 changed how tool calls before the think close are returned, the recipe harness's structured plan ran on the NVFP4 release served from this Ripple code: structured 18/18, drafted == plain 12/12, tools 5/5.

Still waiting on #2171. #2260 is restacked on this head (938b8aef0).

[ashhart]

Thanks @BobClawblaw. The rebase keeps main's masked_values suite, metal_main.zig is at 597 and reply_text.zig at 600, and the CHANGELOG line sits under Unreleased. Nothing more is needed from you for now; this still waits on #2171.

[BobClawblaw]

Rebased onto main c70d67bb2 with #2260 on top: #2231 is now c244847b6 and #2260 8843987e8. CHANGELOG.md was the only conflict, and git range-diff shows every other commit unchanged. The CI set passes on both heads (fmt, lean 0 new, test, golden, kernel contract, security, release paths, fuzz, dev-tools), and tf-kernel-contract --only ricursive gives 75/75 with 25 controls red. Both bodies are updated. Main has since added four DeepSeek prompt commits; against them the branches conflict in CHANGELOG.md only.