Repository navigation
EPIC E6 0.73: Colibri CRUX — expert streaming, auto-fit planner, tiny-MoE fixtures #6481
Description
owner: aprender-w5032 (set 2026-10-30 by aprender-cop per operator ruling N27, noah-rulings.md:574; picks up children only as batch constituents after its current timed task)
Spec: APR-EPIC-001 §2 · Epic: E6-colibri · Train: 0.73 (CH-3 may land 0.74) · Parent: #3999 E6 Runs Everywhere (runtime/memory capability); family work stays with #4001 E7 · Source: JustVugg/colibri v2.0.0 (Apache-2.0)
Theme
CRUX: what colibri (a C engine that runs MoE models larger than RAM by streaming experts from disk) has that aprender lacks, ranked by value ÷ cost, and the three children to pull first. All code is Rust (or pv contracts); nothing is ported in Python or C.
done_when
Each item is owned by one child and has a receipt in evidence/.
- (CH-3) On a Qwen3-MoE 30B-A3B Q4_K_M GGUF, with the process limited by a cgroup
memory.maxset to 60% of the GGUF size,apr runcompletes and decodes at ≥ 50% of the unlimited mmap baseline's tok/s (median of n=5, 1 warmup, same host, host and commit recorded). Output is token-exact against the in-RAM path on the tiny-MoE fixture (CH-2). Control: today's mmap path is measured first under the same cgroup limit, and CH-3 must either beat its tok/s by ≥ 10% (rule 3) or hold peak RSS below it where the control does not. Without that control the item can pass with no code change. Peak RSS and expert-cache hit rate are in the receipt. - (CH-1)
apr serve planchooses quant, context and cache size from a RAM/VRAM/disk budget. A fixed matrix of ≥ 3 budgets has an expected plan each, checked in CI, plus a planted over-budget case that must be refused (the falsifier). - (CH-2/CH-3) A technique counts as adopted only with a one-variable A/B record: ≥ 10% median improvement on its metric, n ≥ 5, non-overlapping min–max ranges, quality gate unchanged. Otherwise it moves to Parked.
Method (CRUX)
Read-only inventory of colibri's docs and headers, checked against aprender origin/main. Every colibri number below is colibri's own claim and has not been replicated; no child may cite it as an aprender result. Nothing is translated line by line: ideas are re-implemented in Rust. A file that copies colibri expression (tables, constants, schemas) carries a header naming colibri and its changes, and the crate ships colibri's LICENSE and NOTICE text (Apache-2.0 is on deny.toml's allow-list).
Ranked candidates
| # | Technique (colibri source) | aprender today | Value | Cost | Pull? |
|---|---|---|---|---|---|
| C1 | Expert streaming + RAM-bounded expert cache: async pread, optional O_DIRECT/io_uring, dense trunk resident (docs/tuning.md, c/pin_pool.h). colibri's README claims a 744B model at 9.9 GB resident. |
ABSENT as an explicit bounded cache. Qwen3-MoE reads experts zero-copy from mmapped GGUF (gguf/qwen3_moe_load.rs), so residency is left to the OS page cache. A CUDA path exists; #3714 is still open. |
Very high | L | CH-3 |
| C2 | Route trace + learned pinning + offline replay simulator (c/route_trace.h, .coli_usage, c/tools/residency_sim.py, docs/routing-telemetry.md) |
ABSENT (moe/ has capacity routing only) |
High; picks C1's policy offline before any wiring | M | CH-2 |
| C3 | Resource planner that fits (c/resource_plan.py, cgroup-aware RAM, partial GPU-chain fit, coli plan) |
PARTIAL: serve_plan.rs checks fit and max batch; it does not choose quant, context or cache |
High | S–M | CH-1 |
| C4 | Test pattern: tiny fixtures + oracles + a deliberately broken *_bad fixture (c/tools/make_*_tiny.py, qwen36_tiny_bad) |
PARTIAL (fixtures, llama.cpp parity; no tiny MoE with a negative fixture) | High; unblocks C1/C2 | S | CH-2 |
| C5 | Speculative: MTP heads + prompt-lookup, one verify forward (docs/speculative.md) |
PARTIAL: speculative.rs is wired only in cbtop_measure_batch.rs, not in run/serve; MTP absent |
Med–high, may be cheap | M | Parked #1 (re-rank after CH-1) |
| C6 | Quant format registry + tensor stamp (docs/FORMATS.md, born of a fmt=6 collision) |
Overlaps #4539 KREG-001; #3433 (typed TensorStorage) is the nearest sibling on tensor self-description |
Med (design input) | S | Parked #2: comment on #4539 |
| C7 | One-variable experiment manifest (c/experiment_manifest.py) |
ABSENT as a schema | Med | S | Parked #3 (CH-2/3 use its fields) |
| C8 | KV prefix reuse, persistence, FP8 / PolarQuant (c/kv_prefix.h, kv_persist.h, kv_fp8.h, kv_tq.h) |
PARTIAL (prefix cache exists) | Med | M | Parked #4 |
| C9 | Vulkan backend and testable hybrid-split math (c/hybrid_split.h) |
PARTIAL: wgpu model/backend exist; Qwen3-MoE wgpu forward pending (M-GPU-MOE-2.0) | High, very large | XL | Parked #5: study hybrid_split.h, do not port |
| C10 | Lossless rANS int4 tier (c/rans.h; colibri claims ≈0.76× the source packed bytes, byte-exact) |
ABSENT | Low–med | M | Parked #6 |
| C11 | expert_atlas measurement traps (colibri reports --topp hides 38% of distinct experts; rejected drafts get counted; usage accumulates across runs; within one run an expert firing 38 times is one observation, not 38) |
n/a | Med | S | Folded into CH-2 acceptance |
| C12 | One-step installer + doctor/status |
PARTIAL (apr pull) |
Med, outside the engine | M | Parked #7 |
| C13 | Model families (GLM, Kimi, DeepSeek V4, MiMo, Qwen-Image, GLiNER) | mostly ABSENT | per family | per family | Parked: E7. Qwen-Image weights are non-commercial: no fixture, CI asset or default model. |
Children (at most 3 open; opened one at a time, in this order)
Slots: this epic is one issue. Each child is opened only by closing or parking one other open issue in its place (APR-EPIC-001 rules 5 and 7, as used in #4001's Parked list), named in the child's body. If no slot is freed, the child stays unopened.
- CH-1
apr serve planauto-fit (C3), S–M, stands alone and sizes the expert-cache budget CH-3 needs. Falsifier: planted over-budget config is refused. - CH-2 Tiny-MoE fixture + CPU oracle +
*_badnegative fixture + route-trace format and Rust replay simulator (C2, C4, C11), M. Rust only. Covers CPU paths; GPU-less GPU CI is not claimed (colibri's Lavapipe trick covers Vulkan only). - CH-3 Bounded expert cache + streaming for Qwen3-MoE (C1), L, depends on CH-1 and CH-2. Lossy options (
--topp, cache-aware routing) are out of scope; only token-exact paths gate.
Not doing
- Verbatim translation of colibri C.
- Third-party-derived parts of colibri (a DeepSeek CUDA router adapted from ds4, MIT; Metal GEMV adapted from Swiftlet, Apache-2.0; the qpack reader; DeepGEMM and CUTLASS fetched at build time) are not read or used by any child.
Parked
C5 speculative/MTP wiring · C6 comment on #4539 · C7 experiment manifest · C8 KV persist/FP8/TQ · C9 Vulkan (study only) · C10 rANS tier · C12 installer · C13 families (E7)
Quorum
Reviewed before filing by three independent review lanes (Opus 5.5; the author was Sonnet 5.5, so this is a same-vendor review). Round 1: three FAIL, all required edits applied. The final verdicts are in the first comment.
Activity
noahgift commentedon Oct 27, 2026
Quorum receipt (pre-filing)
Three independent Opus 5.5 lanes, author Sonnet 5.5 (same-vendor review; agy not used).
All required edits were applied. Open note from lane 2 (optional): the epic issue itself adds one open issue; a named issue to close or park in exchange is still to be chosen by the owner.