[BobClawblaw]

The qwen27 Metal family serves qwen3_5_moe checkpoints. That covers Ornith-1.5-122B, a Qwen3.5-MoE 122B-A10B with a vision tower, on one M5 Ultra, with the incoai DFlash2 drafter, images, the prompt cache and several streams. Context stops at the checkpoint's 262,144 tokens.

Weights: the publisher's own 4-bit MLX checkpoint, ornith-ai/Ornith-1.5-122B-MLX-4bit at revision 06a57aba4fba, served exactly as published: affine 4-bit in groups of 64, with the router and the shared-expert gate at 8 bits. No conversion of ours is involved. The bf16 release (245 GB) does not fit a 256 GB Mac beside its caches, and the publisher's MLX-8bit (131 GB) is a possible follow-up.

Commits (on main fdd8b6d)

1. qwen27: 32 query heads over 2 KV heads on main's attention Geometry. Geometry.check admits 32 over 2 beside main's group of 6 (24 over 4, and 12 over 2 on a rank of qwen27: rank-slice the 27B family and gather the whole embedding table #1433's split); 16 over 1, Ornith's rank, is refused. The kernels take the group as Q27_G/Q27_KVH defines, which default to the 27B's 6 and 4. tf-qwen27-attention-precision gains --heads, --kv-heads and --simd-prompt, which runs the pre-M5 prompt kernel on this GPU.
2. qwen27: serve qwen3_5_moe on Metal.
   • Config: routed fields are read, with an absent norm_topk_prob meaning renormalized weights.
   • Widths: each routed member is admitted on its own. The shared-expert gate shares the router's matrix, so it must share its width. Routed experts are admitted only at the widths the kernel contract checks them at, 4 and 8 bits in groups of 64; any other width, including a group of 32, is refused with UncheckedRoutedWidth before a buffer is sized, and q27_rows_extra is built at those two widths only. A shared expert quantized unlike the routed experts is refused at admission with SharedExpertWidth.
   • Refusals: --speed-up and --learn are refused for qwen3_5_moe by name.
   • Routed block: moe_gpu.zig, moe.metal and moe_rows.metal hold the router, the top-k, the experts, the SwiGLU and the combine. Every prompt chunk runs on the tensor units; decode rows run q27_rows_extra, core's row loop with the shared expert as one more matrix.
   • Prompt chunks are 1,024 rows for routed checkpoints on every chip (prompt_rows.requested takes whether the checkpoint is routed); the dense 27B keeps main's defaults, 2,048 rows on M5 and 512 before it.
   • DFlash2: drafters of up to eight taps and a block of 16, with the tap planes sized from the drafter: the keep in qwen27_keep.zig gathers the drafter's planes, and prompt_memory.Fit counts them (planes) instead of the 27B's five.
   • Memory: the routed block's buffers are part of gpu_frame.allocationSizes, so main's prompt-memory fit admits streams with them counted.
   • Host: qwen3_5_moe is registered, and plain rounds compete with drafted depths on routed models.
   • Neural Engine: the share is refused for routed checkpoints.
   • One Mac only: config.split refuses routed checkpoints (BadShard), so qwen27: rank-slice the 27B family and gather the whole embedding table #1433's rank slice serves the dense 27B alone; nothing of the experts or the router is split.
   • Admission: native/metal.zig routes qwen3_5_moe to the 27B engine's open, beside main's qwen27_engine.owns for qwen3_5; metal.zig stays at 569 lines.
3. kernel contract: the routed block's kernels go through gates 1-5, and the prompt path's core.affine_mm launches at Ornith's widths are registered (the router at q8 g64; the expert gathers and gather-scatters at q4 and q8 g64).

Outside the family folder the PR touches only the contract's registry and tests. It also adds two profile classes (route, experts), a re-export of core.affine_mm, the hub's model-type title and the Metal family list. Core's row projection is unchanged.

Checks (M5 Ultra)

All on the head 22f1026 (main fdd8b6d), with the publisher's checkpoint unless named:

• CI checks: zig build test, test-kernel-contract, test-golden, test-server-security, test-pull-security, fuzz-cortheon, dev-tools and zig fmt --check pass; tools/zig/lean_check.py finds 424 problems, as on main (0 new). zig build test has one failure, contract image: every registered CUDA launch and control selects its own image (unknown image name: minimax_m3); main fdd8b6d fails it the same way (zig build test-cuda-host), so it predates this PR.
• Kernel contract: --only qwen27: SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems. --only affine_mm: SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 4 known findings; 3 controls: 3 red as planted, 0 missed; 0 limit problems (the 4 known findings are main's dense q4 one: an infinite input gives NaN where MLX keeps the infinity).
• Pre-M5 prompt kernel at a group of 16: tf-qwen27-attention-precision --simd-prompt --heads 32 --kv-heads 2 gives max error 3.906e-3 and mean 6.18e-5 against float64 (queries in [-1, 1)); at 24 over 4, 3.906e-3 and 6.14e-5, which are 993cebd's M3 Ultra numbers. It ran on the M5's simdgroup matrices, since no Mac before M5 was available.
• Resumed equals fresh at any chunk boundary: tf-qwen27-run --chunk 1024 --compare-chunk 13 --seams 1024 over 8,192 tokens gives byte-equal prompt state, decode state and seams at every 1,024 rows, and the runner reports no leaks.
• Unchanged by the rebase: the same run's 64 greedy tokens and prompt digest are identical to the previous head 078e224's, run in the same session (prompt 2,478 against 2,475 tok/s).
• Dense 27B A/B against main fdd8b6d (Qwen3.8-27B-MLX-4bit): tf-qwen27-run on an 8,192-token prompt, 256 greedy tokens, 2,048-row chunks, run main, head, head, main: identical tokens and prompt digest in all four; prompt 1,646-1,658 tok/s on both, decode 49.9-50.2 tok/s (main's first run, the session's first load, 43.2).

Speed (the recipe harness, the publisher's checkpoint, four streams of 262,144 tokens, 32 GiB prompt cache, drafter on)

Measured on the previous head 078e224 and not rerun after this rebase; the family's code is unchanged and the runs above give identical tokens. One stream, 256-token greedy replies, tok/s, measured on 31 Oct within the same hour:

Build	prose	code	structured	list
078e224 (the previous head)	132.5	189.6	164.7	198.7
a410eb5 (the head before)	94.0	157.7	133.3	150.0

The family's diff is the same in both, so the gain is main's, from the 27B work it rebased onto. Two and four streams, from the previous measurement on 8aeabea05 and not rerun here:

Streams	prose	code	structured	list
2	193.2	276.7	235.9	306.2
4	234.0	316.7	284.1	354.1

Cold prompts on 8aeabea05, thinking off: 2,210 tokens at 2,514 tok/s, 8,772 at 2,484, 35,011 at 2,164 and 139,759 at 1,336.

Serving checks on 078e224 (the harness's quick plan): seeded sampling equals its repeat and the undrafted reply (9/9), and structured output obeys its schema (18/18, drafted equal to plain 12/12). A resumed turn equals the same turn again and a cold one after a restart. Sixteen streams together match the same requests alone (12/12), and a 60-turn soak beside a decoding stream matches too (60/60). Drafts (12/12), stop strings, max-tokens, streaming, thinking, copying and multi-turn recall all pass. Images: 4/4 single requests right and sixteen together equal to alone (16/16); the suite's two video cases are refused, since the family has no video path. One named-tool case with thinking on fails as before (it repeats its call). The image-URL check fails because that server ran without --vision-urls; the recipe now serves with it, and there (on a410eb5) a public HTTPS image is answered while loopback and plain-HTTP URLs get HTTP 400.

Quality

Not rerun for the rebase. The recipe harness's quality suites on aa472b6 with the publisher's checkpoint, on the server above (greedy, one run each). Thinking off: tools 55/60, json 30/30, IFEval 120/150, GSM8K 245/250, MGSM 220/240, MMLU 298/342 and MATH-500 438/500. Drafts (12/12), streaming, multi-turn, stop strings, the thinking toggle, copying and max-tokens all pass. Thinking on, with a 49,152-token reasoning budget: GSM8K 244/250, and the harness's ten word problems are all right, none cut at 50,176 tokens.

Recipe: https://github.com/BobClawblaw/ornith-15-122b-mlx-m5-ultra

[ashhart]

Thanks @BobClawblaw for a complete Ornith port, for checking the routing layer by layer against mlx-vlm, and for correcting the norm_topk_prob default. We are carrying all fifteen commits onto current main ourselves, kept as yours, and a separate commit of ours will fix what the review found: the expert_sums buffer is sized for group 64 (zig/src/families/qwen27/gpu_frame.zig:45) although group 32 is admitted and needs twice the space, main's keep buffers and residency-owner mock need adjusting for Ornith's two KV heads and the new affine_mm import, and the shared projections' format, the shared gate's own width and the YaRN factor get checked at load. The same commit adds kernel contract cases with planted controls for the five routed kernels in moe.metal and the tf_row_projection_indexed_extra variants, including the sorted prompt path against the indexed one around 16 rows. Then it goes to the Metal kernel contract and served-equality runs on M5 and on a chip without tensor units, and lands. You need not rebase; if you are in the middle of a change, please tell us so we do not collide.

- Synthema, Ash's AI Assistant.

[ashhart]

Thanks @BobClawblaw. This no longer rebases onto main: the first commit conflicts in build/qwen27.zig, attention.zig, decode_round.zig and tests/qwen27/run.zig, because main now passes attention shape through attention_plan.Geometry (which accepts only 24 query heads over 4 KV heads and refuses 2) and added a pre-M5 prompt kernel (attention_prompt_sg.metal). Ornith's 32 heads over 2 means qualifying that kernel at a group of 16, so please rebuild the head/KV-head change on main's Geometry. Also:

1. Register the new kernels with the Metal kernel contract: tf_row_projection_indexed_extra (shared core) and the moe.metal route/sort/gather/SwiGLU/combine kernels.
2. The dense 27B's default path changes: qwen27_served.zig sizes the drafter ring for 8 taps instead of 5 and taps.zig always allocates 8 planes, which raises memory per stream and can lower admitted streams. Please size both from the drafter's tap count.
3. YaRN belongs in its own PR: it changes dense-27B context limits, parses rope_scaling for dense checkpoints, and rewrites the rope kernel, so it needs its own token-hash checks.
4. Remove the debug switches (TF_LAYERS, TF_DRAFT_CHILDREN/TAU/EDGE, the logits/route dumps).
5. Prompt chunks of 16+ rows on M5 route and run experts on tensor units while decode uses the row kernels, so router logits can differ in the last bit and pick different experts. Resumed must equal fresh wherever chunk boundaries fall; please make routing path-independent or show it is. The Neural Engine share also needs qualifying for this model.
6. Smaller: gpu_weights.zig leaks two name strings per layer; the shared-expert gate loads at the router's width; the moe.metal comment says "/ topk" but divides by topk + 1.

[BobClawblaw]

Thanks for the review. The branch is rebuilt on main (eea9dc9, 1.1.6) as three commits, head aa472b6, and each point is addressed below. The PR now serves the publisher's own checkpoint, ornith-ai/Ornith-1.5-122B-MLX-4bit at revision 06a57aba4fba, exactly as published: no conversion of ours is involved. Figures are from an M5 Ultra. The chunk check and the serving and speed runs used the head with that checkpoint. The rest used 0f0337736, which differs only by main's two 1.1.6 release commits.

Rebase and attention shape. The head/KV-head change now sits on main's Geometry: check() admits 32 over 2 beside 24 over 4, and the kernels take the group as Q27_G/Q27_KVH defines whose defaults are the 27B's 6 and 4. tf-qwen27-attention-precision gained --heads/--kv-heads and --simd-prompt, which runs attention_prompt_sg on this GPU's simdgroup matrices. 128-row prompt chunks against float64:

kernel	24 over 4	32 over 2
attention_prompt_sg	max 3.906e-3, mean 6.14e-5	max 3.906e-3, mean 6.18e-5
attention_prompt (M5)	max 4.093e-3, mean 6.24e-5	max 4.068e-3, mean 6.28e-5

The 24-over-4 simdgroup row matches the M3 Ultra figures 993cebd quoted. I had no Mac before M5 to run it on.

1. Kernel contract. The routed block's kernels are registered. tf_row_projection_indexed_extra moved out of core: it is now q27_rows_extra in kernels/metal/qwen27/moe_rows.metal, built on core/row_projection.metal's row loop with core's kernels cut, as GLM builds its rows. Core's row projection is therefore unchanged. Gates 1-4 cover q27_rows_extra (4 and 8 bits), q27_route, q27_route_sort, q27_route_gather, q27_expert_glu and q27_expert_combine (in place and from its slot). tf-kernel-contract --only qwen27: SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 0 known findings; ... 0 limit problems. Gate 5 records the launches. The moe.metal kernels compile for M1 and M2 at 1,024 threads (--offline), and the row kernel reserves its 64. The offline run's one finding, glm_rows_bf16_nax, is the same on main.
2. Taps. The tap planes start at the 27B's five and grow only when a drafter of more taps opens. slotCost reads the drafter's own tap count. A/B on Qwen3.8-27B-MLX-4bit against main d0948b6:
   • tf-qwen27-run on an 8,192-token prompt: tokens and prompt digest identical, 1,603 and 1,600 tok/s.
   • Served with the DFlash2 drafter: both builds admit four streams of 32,768 tokens. Eight greedy replies, drafted and plain, are identical.
   • Decode times are equal within 0.3% over alternating runs.
3. YaRN is out. The context stops at the checkpoint's 262,144 tokens, and no YaRN PR will follow.
4. Debug switches are removed: TF_LAYERS, TF_DRAFT_CHILDREN/TAU/EDGE and the logits dump. The route dump had already gone. The PR adds no environment reads.
5. Routing path. Every prompt chunk on tensor units now routes and runs its experts on the tensor-unit path, whatever its rows, so a row routes alike wherever its chunk starts. Decode windows keep the row kernels, as decoded spans replay them. tf-qwen27-run --chunk 1024 --compare-chunk 13 --seams 1024 on the publisher's checkpoint over 8,192 tokens gives byte-equal prompt state, decode state and seams at every 1,024 rows. The previous head fails the same check with PromptStateOrLogitsDiffer. The Neural Engine share is not qualified for this model, so a routed checkpoint now refuses it and says so; prompts stay on the GPU.
6. Smaller items.
   • The two name strings a layer are freed. The previous head's runner reported 96 leaked allocations, two per layer over 48 layers, and this one reports none.
   • The shared-expert gate loads at its own width. Because it shares the router's matrix, admission refuses a checkpoint that quantizes the two differently.
   • The gather comment now says topk + 1.

Speed on the head is in the updated description (the ruler at one stream drafted 94.1/157.7/132.9/149.8, cold prompts 2,499 to 1,334 tok/s from 2k to 140k). zig build test, test-kernel-contract, test-golden, test-server-security, test-pull-security, fuzz-cortheon and zig fmt --check pass; tools/zig/lean_check.py finds 433 problems, as on main.

[BobClawblaw]

Rebased on main 3f408b6 (head 6e84105), with two fixes. The checks below ran on 8aeabea05, which is the same branch on 03c2820; main's one newer commit changes only qwen4's host tests.

• zig build dev-tools failed to compile three qwen27 tools: zig/tests/qwen27/dflash_graph.zig still built a five-element tap list where the drafter config now holds eight. The literal is padded, and tap_count stays at five. The fix is in the qwen3_5_moe commit.
• The rebase put native/metal.zig at 602 lines. The 27B's admission and open (qwen27Engine, openQwen27 and their test) move unchanged into native/metal_qwen27.zig. tools/zig/lean_check.py now finds 428 problems, against main's 429.

On the head: zig build test, dev-tools, test-kernel-contract, test-golden, both security tests, fuzz-cortheon and zig fmt --check pass. tf-kernel-contract --only qwen27 gives the same SUMMARY. On the publisher's checkpoint, the chunk check (--compare-chunk 13 --seams 1024) is byte-equal, with greedy tokens and prompt digest identical to the previous head. The harness's serving checks, ruler and cold prompts are within run-to-run noise of the previous head, and the description now quotes the new figures. The desktop job's failure is app/electron/agent/bot-vm/guest.test.ts ("VM tool refused: [Errno 1] Operation not permitted"), which this PR does not touch.

[ashhart]

Thanks @BobClawblaw. Nearly everything from our 26 Oct review is done:

• the 32-over-2 heads on Geometry;
• tap planes sized from the drafter;
• YaRN and the debug switches out;
• the leaks and the shared-gate width fixed;
• tensor-unit prompt chunks taking one routing path whatever their row count, with decoded spans replaying through the row kernels.

We rebased it onto main b30d5b9d3 (clean). zig build test passes, and lean is one under main.

Still to do:

1. expert_sums is sized for group 64 (gpu_frame.zig:45, flagged on 22 Oct). checkRouted admits group 32 for routed experts on tensor units (widths.zig:218), and rowSums writes K/32 sums a row there. So a g32 checkpoint overruns the buffer on any tensor-unit prompt chunk. Please size it for the smallest admitted group, or refuse group 32.
2. Contract. q27_rows_extra is built and admitted at 2, 3, 5 and 6 bits, but only q4 and q8 are registered. The prompt path's affine_mm launches (the router at q8 g64, the expert gathers at q4 g64) aren't in the records. Please register them, or refuse the widths you haven't checked.
3. Fidelity. Please run teacher-forced against a reference (mlx-vlm or transformers) on the publisher's weights, and list each top-token miss with its contenders and margins.
4. Licence. Please state the licences of Ornith-1.5-122B and the incoai DFlash2 drafter. The GLM drafter from incoai is CC BY-NC-ND 4.0.
5. Smaller.
   • A shared-expert width mismatch passes admission and only fails at the first decode (moe_gpu.zig:55).
   • --speed-up and --learn are silently ignored for qwen3_5_moe, because refusal() checks "qwen3_5".

We'll run it on an M5 Ultra and on an M3 Ultra for the pre-M5 prompt kernel at a group of 16.

[BobClawblaw]

Thanks for the review, and for rebasing and running it. All five points are done on a410eb5, rebased onto main a05d620:

1. expert_sums and group 32. The routed block now admits only the widths the kernel contract checks: 4 and 8 bits in groups of 64 (widths.zig, routed_bits and routed_group). A group-32 routed checkpoint is refused at admission with UncheckedRoutedWidth, before any buffer is sized, so expert_sums (sized for group 64) can't be overrun. The host test the routed block admits the publisher's widths and refuses those its kernels are not checked at covers the publisher's 4-bit layout, uniform 8-bit, a g32 checkpoint and one 5-bit expert projection.

2. Contract. q27_rows_extra is now built only at q4 and q8, from the same routed_bits list admission uses, so no width is compiled or admitted unchecked. The prompt path's affine_mm launches are registered: the router at q8 g64, at a 1,024-row chunk and an odd 65-row tail with 320 rows (256 experts plus the shared gate, padded), and the expert gathers and gather-scatters at q4 and q8 g64. Gate 1 for the gathers plants its NaN and infinities in one expert, so the others' outputs stay finite and the class check proves something. On an M5 Ultra:

   --only qwen27     SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 0 known findings; 0 controls: 0 red as planted, 0 missed; 0 limit problems
   --only affine_mm  SUMMARY 8 kernels: 8 pass gates 1-4, 0 fail, 4 known findings; 3 controls: 3 red as planted, 0 missed; 0 limit problems
   

   The 4 known findings are main's dense q4 one, carried by the q4 gathers: an infinite input gives NaN where MLX keeps the infinity (the +128 code offset). The three controls (an expert's last row dropped, a tile reading its neighbour expert's weights, outputs left in sorted order) each go red. Gate 5: the affine_mm records (affine_gather_records.zig) launch 128 threads on tensor units, which have no M1 or M2 build. The routed block's own moe.metal kernels compile for M1 and M2 at 1,024 threads (tf-kernel-contract --offline); that run's one finding, glm_rows_bf16_nax, is main's record and kernel, untouched here.

3. Fidelity. Reference: MLX 0.32.2 with mlx-lm 0.31.3 running the publisher's files (ornith-ai/Ornith-1.5-122B-MLX-4bit @ 06a57aba4fbae218efc9716bcdf7d6b3035331c8) on the same M5 Ultra. The bf16 release (244 GB) does not fit beside anything on one 256 GB Mac, so the reference runs the same 4-bit weights the PR serves. Six chat prompts (code, arithmetic, an explainer, German, Chinese, a list; thinking on, as the template renders it) each decode 256 greedy tokens in MLX one token at a time, keeping its top 8 at every position. tf-qwen27-run --teacher then feeds those tokens to the engine and records its top two at every position.

• Engine against MLX: top-1 equal at 1,498 of 1,536 positions (97.53%), 38 misses.
• MLX against itself: one teacher-forced prompt pass over the same tokens (MLX's prompt kernels) against its one-token steps (its decode kernels) agrees at 1,500 of 1,536 (97.66%), 36 misses.
• The misses look alike. MLX's decode margin at the engine's misses has median 0.50 and maximum 3.09 logits, 8 above 1. At its own prompt-pass misses: median 0.41, maximum 3.88, 7 above 1. 15 positions miss in both lists, and at 14 of them MLX's own prompt pass picks the engine's token.

So on the publisher's weights the engine is as far from MLX's decode as MLX's prompt pass is. The larger flips (about 3 logits, at low-confidence positions such as p0 position 159 and p4 position 240) occur in MLX against itself too. Every miss, with both sides' contenders:

The 38 positions where the top tokens differ

prompt	pos	MLX top-2 (logits)	margin	engine top-2 (logits)	margin	MLX's prompt pass picks
0	34	. 28.250, ␣with 27.875	0.375	␣with 26.250, . 25.625	0.625	the same as MLX's decode
0	159	, -3.469, ]) -6.562	3.094	]) -4.656, , -8.188	3.531	the same as MLX's decode
0	179	␣result 10.250, ␣i 9.812	0.438	␣i 11.938, ␣result 10.812	1.125	the engine's pick
0	251	␣( 12.250, ␣from 11.438	0.812	␣from 13.562, ␣( 13.250	0.312	the same as MLX's decode
1	2	␣work 19.625, ␣solve 19.125	0.500	␣solve 15.812, ␣work 14.375	1.438	the same as MLX's decode
1	8	Train 14.562, First 14.000	0.562	First 14.938, Train 14.688	0.250	the same as MLX's decode
1	21	␣travels 16.000, ␣speed 14.625	1.375	␣speed 15.375, ␣travels 14.875	0.500	the same as MLX's decode
1	65	␣let 36.000, ␣what 36.000	0.000	␣is 35.500, ␣let 35.500	0.000	the same as MLX's decode
1	67	␣think 33.750, ␣re 33.000	0.750	␣re 33.000, ␣think 33.000	0.000	the engine's pick
1	96	␣to 26.875, ␣of 26.750	0.125	␣of 26.750, ␣to 26.375	0.375	the same as MLX's decode
1	198	Now 32.750, Train 31.875	0.875	Train 31.375, Now 30.250	1.125	the engine's pick
2	145	␣for 27.000, ␣maximum 24.625	2.375	␣the 25.625, ␣for 24.875	0.750	the same as MLX's decode
3	8	␣text 14.438, ␣message 13.688	0.750	␣message 13.562, ␣text 13.438	0.125	the engine's pick
3	75	␣res 21.750, ␣meeting 20.125	1.625	␣meeting 21.750, ␣res 20.375	1.375	the same as MLX's decode
3	91	\n 20.500, ␣( 18.875	1.625	␣( 20.625, \n 19.750	0.875	the same as MLX's decode
3	106	␣an 31.000, ␣informal 30.875	0.125	␣informal 31.250, ␣an 30.625	0.625	the engine's pick
3	118	␣form 27.125, ␣( 26.000	1.125	␣( 26.625, ␣instead 26.000	0.625	the engine's pick
3	122	Sie 4.625, i 4.469	0.156	i 3.125, Sie 3.125	0.000	the same as MLX's decode
3	150	␣nicht 18.750, ␣fehlt 18.125	0.625	␣fehlt 18.375, ␣nicht 18.250	0.125	the same as MLX's decode
3	168	␣this -8.125, : -8.625	0.500	: -13.375, ␣this -13.375	0.000	the engine's pick
3	193	␣→ -7.500, ␣- -8.812	1.312	␣- -7.188, ␣→ -8.062	0.875	the same as MLX's decode
3	232	␣morgen 23.125, ␣die 22.250	0.875	␣die 22.875, ␣morgen 22.750	0.125	the same as MLX's decode
4	9	意义 39.500, 、 39.250	0.250	、 39.250, 意义 39.250	0.000	the engine's pick
4	66	自 22.375, 流经 22.375	0.000	流经 22.625, 自 22.375	0.250	the engine's pick
4	86	广阔 22.875, 广 22.750	0.125	约 21.125, 广 20.625	0.500	广
4	143	促进了 19.375, 促进 18.875	0.500	长江 19.875, 促进了 19.625	0.250	the same as MLX's decode
4	160	的重要 13.438, 的重要性 13.438	0.000	是 16.000, 的重要 14.500	1.500	the same as MLX's decode
4	197	水利工程 16.750, 工业 16.625	0.125	工业 17.500, 水利工程 17.250	0.250	the same as MLX's decode
4	240	\n 5.344, \n\n 2.547	2.797	\n\n -4.812, \n -7.781	2.969	the engine's pick
5	38	. 29.375, ␣accordingly 28.500	0.875	␣accordingly 28.625, . 27.875	0.750	the same as MLX's decode
5	65	␣global 21.375, ␣caches 21.250	0.125	␣caches 21.750, ␣global 21.375	0.375	the engine's pick
5	72	␣usually 26.625, ␣fixing 26.375	0.250	␣fixing 24.625, ␣often 24.500	0.125	the engine's pick
5	141	/module 21.625, ␣caches 21.500	0.125	␣caches 22.500, /module 19.875	2.625	the engine's pick
5	152	␣restarting 18.750, ␣using 18.250	0.500	␣using 18.250, ␣restarting 17.125	1.125	the same as MLX's decode
5	163	␣items 23.000, ␣tips 22.375	0.625	␣tips 23.125, ␣answers 22.750	0.375	the same as MLX's decode
5	176	␣first 32.000, ** 31.875	0.125	** 31.500, ␣first 30.625	0.875	the engine's pick
5	206	␣references 22.625, ␣timers 22.500	0.125	␣timers 23.125, ␣references 22.625	0.500	the same as MLX's decode
5	235	␣collection 28.125, ␣store 27.500	0.625	␣store 28.750, ␣collection 28.375	0.375	the same as MLX's decode

You named mlx-vlm or transformers. Transformers can't load the MLX 4-bit files, so I replayed the same six histories through mlx-vlm 0.7.10's qwen3_5_moe language model (text only, MLX 0.32.4), one token at a time. It agrees with mlx-lm's top token at 1,500 of 1,536 positions, about as often as the engine does (1,498). At the engine's 38 misses, mlx-vlm picks the engine's token at 16 and mlx-lm's at 22. The engine's top two at all 1,536 positions are identical on a410eb5 and on the build the table came from.

4. Licences. As each repository's model card states at the pinned revision:

• ornith-ai/Ornith-1.5-122B @ 7681ef608624e7532d8164128bcffdbdbaf1b1af: MIT (license: mit). The card's license_link points to a LICENSE file in that repository, which is not there (404, at the revision and on main); the card's metadata is the only statement.
• ornith-ai/Ornith-1.5-122B-MLX-4bit @ 06a57aba4fbae218efc9716bcdf7d6b3035331c8, the checkpoint the PR serves: MIT, with the same license_link to the release.
• incoai/Ornith-1.5-122B-DFlash2 @ a5ed7396f5f9e5b1abf7ff50b7899a3186961432, the drafter: Apache-2.0 (license: apache-2.0, base_model: ornith-ai/Ornith-1.5-122B). The card has no other licence terms, and main (9baefc1bec85c08193ab442e0ca5f7057669b61d) states the same. Unlike incoai's GLM drafter, it is not CC BY-NC-ND.

5. Smaller.
   • A shared expert whose width differs from the experts' is now refused at admission (SharedExpertWidth, naming the layer and module), not at the first decode. Host test included.
   • --speed-up and --learn are refused for qwen3_5_moe by name ("Qwen3.5-MoE"), with tests. The --learn refusal no longer points routed checkpoints at the dense 27B's --learn-conversations.

Checks on a410eb5, rebased onto main a05d620 (the rebase onto main's TF_Q27_PROFILE work had two conflicts, both resolved keeping both sides): zig build test, test-qwen27 and test-kernel-contract pass; zig fmt --check is clean; lean 425 problems, 426 on main, 0 new; both contract runs above; the teacher-forced top two at all 1,536 positions are identical to the run on 1ff172109, the same commits on main 1b9a987. On 1ff172109, the dense 27B A/B against main 1b9a987 (Qwen3.8-27B-MLX-4bit, tf-qwen27-run, an 8,192-token prompt and 256 greedy tokens): identical tokens and prompt digest, prompt 1,417.6 against 1,415.9 tok/s.

[ashhart]

Thanks @BobClawblaw. All five points are answered in a410eb5, and sorry this has waited three days. G32 is refused, the widths are registered, the teacher-forced numbers sit next to MLX's, and the licences and refusals are done.

Main has moved under it since then. #1686 (32-row lane windows), #1704 (unrolled Metal tiles) and #1726 (the M5 lane loop by default, and a tail hook in the forward) all landed in the 27B family. A rebase now conflicts in gpu_frame.zig, model.zig, native/metal.zig, qwen27_lanes.zig and qwen27_served.zig. That's the core of the 27B, and you know your side of it better than we'd guess it, so could you rebase onto current main?

While you're at it, please author the commits as 288991054+BobClawblaw@users.noreply.github.com, the form your other PRs use, rather than the older address without the number.

Once it's pushed, we'll gate it and run the M5 Ultra and M3 Ultra cells: Ornith against MLX's own numbers, plus the 27B against main to show nothing moved there. That includes the pre-M5 prompt kernel at group 16.

[BobClawblaw]

Thanks @ashhart, and no worries about the wait. Rebased onto main 94b90c9 and force-pushed as 078e224, with all three commits authored as 288991054+BobClawblaw@users.noreply.github.com.

Fitting onto main's moves:

• The 27B engine choice stays in qwen27_engine.zig; metal.zig opens qwen3_5_moe beside it, and the separate metal_qwen27.zig is gone.
• The keep in qwen27_keep.zig gathers the drafter's tap planes rather than five, and prompt_memory.Fit counts those planes and the routed block's buffers (now part of gpu_frame.allocationSizes).
• Routed checkpoints keep 1,024-row prompt chunks on every chip; the dense 27B keeps main's 2,048 on M5 and 512 before it.

On the M5 Ultra, on this head: resumed equals fresh with --compare-chunk 13 --seams 1024, the 64 greedy tokens and prompt digest are identical to a410eb5's, and the dense 27B against main gives identical tokens and digest at the same speed. The kernel contract summaries and the group-16 pre-M5 prompt kernel numbers are unchanged. Details are in the updated description.

atc overlap shows akol1's two-Mac 27B series (#1433-#1437) conflicting with this in the same files, so whichever lands second will need a rebase; I'm happy to do it if this one is second.

[ashhart]

Thanks @BobClawblaw, clean rebase: it applies on main as is, lean and fmt pass, and the identities are right. We've added a changelog line. We don't have Ornith's Metal checkpoint on our Macs, so the 122B lands on your receipts. We're running Qwen3.8-27B with DFlash2 on an M5 Ultra, main against this, to show the 27B didn't move, since this touches its engine. Results here. It lands after 1.2.0.

[BobClawblaw]

Thanks @ashhart. Rebased onto current main (060ddc0): head 53c9074, the same six commits.

On main itself the rebase was close to mechanical. Main's qwen27 files have not changed since the PR's base (5fe74eb), so the family commits apply as they were. There were two textual conflicts, both in lists: the Metal record list in core/contract/registry.zig (main added ling_route_records; both are kept) and the CHANGELOG. The 32-over-2 Geometry commit is on main as 08752c8 and was not part of this branch any more. Your two commits (the changelog line and the dflash_runtime plane count) are kept with their trees unchanged, re-committed under my identity because our push hook takes only that one.

The six-file conflict looks like it came from #1817 (lead/qwen-dense-geometry, still open), not from main. I tried this branch on top of #1817 as well. There it conflicts in one place, model.zig supported(). The resolution keeps #1817's geometry.supported(c) for every checkpoint, with our explicit Ornith shape gate beside the dense one:

const dense = qualifiedSize(c) and c.kv_heads == 4 and c.k_heads == 16 and !c.moe();
const routed = c.hidden == 3072 and ... and c.moe_intermediate == 1024;
if (!(dense or routed) or c.vocab != 248320) return error.UnqualifiedQwenShape;
try @import("geometry.zig").supported(c);

If #1817 lands first, I'll rebase onto it with that resolution.

Checks on 53c9074: zig fmt --check, zig build, dev-tools, test, test-golden, test-kernel-contract, fuzz-cortheon, test-server-security/test-pull-security and check-release-paths all pass; lean 424 problems, 424 on main, 0 new.

Live on the M5 Ultra (publisher's 4-bit, 32k x 4, greedy, 1/2/4 streams, the same eight prompts as on 2 Nov): Ornith's replies with DFlash2 drafts and without drafts, and the dense Qwen3.8-27B's replies with drafts, are identical, token for token, to the reviewed head's. Speed is level within the noise of a busy machine (Ornith drafted: 106.6 / 202.9 / 191.6 tok/s at 1 / 2 / 4 streams, against 111.6 / 201.6 / 192.2 before).

[BobClawblaw]

@ashhart, #1817 has landed, so I rebased #1125 onto it: head e75c89d. In supported(), #1817's geometry.supported(c) now runs for every checkpoint, and Ornith's shape gate sits beside the dense one. The full CI passes, and Ornith's live replies (with and without drafts) and the dense 27B's (with drafts) are identical to before. #1799 is restacked on it (bd524f1).

Main has moved again since, and the only conflict is the CHANGELOG line (#1928 landed above ours); the code merges cleanly. I'll leave the branch as it is, so it doesn't conflict again with every landing. Tell me when you want it rebased for landing, or drop the changelog commit and keep main's.

[ashhart]

Thanks @BobClawblaw. The rebase onto #1817, with Ornith's shape gate beside the dense one, is what we wanted. No need to rebase again for the CHANGELOG; we'll take main's and re-add your line when we prepare it. Since supported() changed, we'll re-run the dense 27B A/B on an M5 Ultra on this head. If it's green, this lands, with #1799 straight after.

[ashhart]

M5 Ultra A/B on this head (e75c89d99) for the changed supported(), dense Qwen3.8-27B MLX 4-bit, base-new-new-base: GREEN.

• Exact: 0 of 20 prompt rows differ.
• Decode: level, one stream at 48.5 → 48.7 (chat, T 0), 51.5 → 51.5 (chat, T 1), 48.8 → 48.7 and 51.6 → 51.6 (code).
• Prompt reading: level or better. 8K reads 1769 → 1855 tok/s; the base arm itself swings between 1675 and 1855, so call it level.

So the dense 27B is untouched by Ornith's shape gate. That was the last thing on our side, so this goes into the landing queue now, with the CHANGELOG conflict resolved there, and #1799 straight after it. Thank you @BobClawblaw.

[ashhart]

Thank you @BobClawblaw. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

[ashhart]

Thanks @BobClawblaw! This is in TensorFold 1.2.3, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.3