v0.3.0: MQ4R + Redline across RDNA

[Kaden-Schutt]

Summary

This draft promotes the accumulated beta staging line toward hipfire v0.3.0.

The release integrates Redline as an in-tree RDNA dispatch and retained-replay substrate, establishes Qwen 3.6 35B-A3B MQ4R as the headline performance SKU, refreshes the model/serving surface, and carries cleanly extracted contributor deltas from the PR backlog.

Contributor work was rebased or cherry-picked as a delta onto the current beta line rather than merged wholesale. That keeps unrelated older code from overwriting the newer modular runtime and Redline implementations.

Headline: MQ4R + Redline

Redline records hipfire's actual decode graph, derives resource dependencies, retains invariant command state, and lowers validated routes through public ROCr queue interfaces. The substrate supports the RDNA family from RDNA1 through RDNA4; optimized routes remain architecture- and workload-specific and fail closed to ordinary HIP dispatch.

Qwen 3.6 35B-A3B MQ4R ordinary AR with Q8 KV:

Architecture	TG128 AR	8-turn average	Final turn	Session health
gfx1100	253.3 tok/s	191.0 tok/s	160.3 tok/s at 18.2K	8/8 clean
gfx1151	115.1 tok/s	92.2 tok/s	82.5 tok/s at 21.3K	8/8 clean
gfx1201	203.9 tok/s	169.5 tok/s	146.7 tok/s at 22.2K	8/8 clean; both recall probes 3/3

Short-context ranges were 253.04-253.48 tok/s on gfx1100, 115.02-115.18 tok/s on gfx1151, and 203.42-204.04 tok/s on gfx1201.

These are ordinary autoregressive results: no MTP, DFlash, speculative acceptance, reduced-output benchmark, or manual clock pinning. The gfx1201 campaign raised the path from approximately 110 to 203.9 tok/s.

User-facing release changes

• Rewrites the README around MQ4R/Redline, the measured cross-arch results, and the actual curated registry rather than mixing runtime architectures with published pull tags.
• Documents all 54 registry entries by their real, fully qualified tags and points users to hipfire list -r as the live source of truth.
• Updates the MQ4R model card description with current gfx1100/gfx1151/gfx1201 measurements.
• Bumps the Rust workspace and lockfile packages from 0.2.1 to 0.3.0.
• Adds the standalone v0.3.0 release history to CHANGELOG.md.

Contributor deltas included

PR	Contributor	Extracted scope
#465	@fivetide	LLaMA Site A attention dispatch and expanded HFQ KV policy
#466	@fivetide	Qwen2 instruct chat-template application
#468	@fivetide	dots.ocr text-only daemon generation
#473	@fivetide	VibeThinker-3B MQ4/MQ6 registry entries
#476	@fivetide	DeepSeek V4 gfx1151 i8-WMMA prefill
#479	@fivetide	MiniMax gfx1151 grouped/dense prefill and fail-closed guards
#480	@fivetide	MiniMax projections through execute_steps
#482	@fivetide	Fused QKV bias across per-row dtypes, isolated from Redline's ABI
#487	@fivetide	Architecture-generic MMQ screening
#496	@HUSRCF	Default-off RDNA3 QKVZA split-tail experiment
#497	@fivetide	Multi-stage runtime and GPU gate-runner Containerfile
#501	@fivetide	DSpark request telemetry and non-fatal pre-warm recovery
#513	@nwoolmer	Native Qwen XML tool calls across CLI, daemon, and cached history
#528	@nwoolmer	DeepSeek V4 DSpark sidecar registration and re-pull discovery
#529	@HUSRCF	Quickstart refresh and historical benchmark labeling

PR #496 remains default-off behind its feature flag; no physical-GPU release claim is made for that experiment.

Additional beta work

• Intentional integration of the current Redline line.
• gfx1100, gfx1151, and gfx1201 MQ4R kernel/replay tuning with architecture isolation.
• In-tree redline-dispatch and redline-rocr crates.
• DSpark telemetry, drafter discovery, and pre-warm recovery.
• Qwen native XML tool-call routing.
• DeepSeek V4 and MiniMax gfx1151 prefill work.
• LLaMA KV dispatch, Qwen2 template handling, dots.ocr text-only serving, generic MMQ screening, and fused-QKV bias.
• Bun/TypeScript, TUI, HTTP, hashing, and dynamic-loading dependency refresh.
• Adoption of clap, safetensors, half, tracing, and proptest.
• Redline compiled-plan/property coverage and cargo-deny CI integration.
• Removal of the stale modular rebase helper and dead AWQ router exclusion.

Validation

Release metadata commit:

• cargo check --workspace --lib
• cargo test --lib --workspace --locked
• python3 scripts/registry_gen.py --check
• bun run typecheck
• bun test: 288 passed, 0 failed
• git diff --check

The registry generator re-probed every curated Hugging Face repository and produced a valid strict superset with 54 models and 38 aliases.

The Rust commands complete with existing unused/dead-code warnings; no new build error was introduced by the release version bump.

Hardware evidence for the headline MQ4R table comes from the recorded beta/Redline product and eight-turn serving artifacts on gfx1100, gfx1151, and gfx1201. This final release-metadata commit did not rerun the physical GPU matrix.

Remaining before promotion

• Run the final release hardware matrix from the exact PR head.
• Preserve the gfx1100/gfx1151 long-session artifacts in the PR or a repository checkpoint.
• Finalize and build the per-family/all-RDNA Linux release assets.
• Change the README's “next release” line to “current release” only when v0.3.0 is tagged.

[HUSRCF]

Possible gfx1100 MQ4R long-prefill regression in beta

Thank your effort in putting together this v0.3.0 promotion PR. While validating the headline Qwen3.6-35B-A3B MQ4R path, I came across a result that might indicate a gfx1100 regression in beta and might be useful to consider as part of the final hardware matrix.

Environment: 2 × Radeon Pro W7900 (gfx1100), HIP 7.2, Q8 KV, and qwen3.6-35b-a3b.mq4r. The model SHA-256 is 4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428.

The matched results were:

• master@5d3683a7 (clean): PASS — 41.3 ms warmup and 132.6 tok/s decode.
• beta@9ffb18da (clean): FAIL — HIP illegal memory access during the first warmup/decode forward.
• beta@9ffb18da (local dirty worktree): STALL — stopped at the same warmup point.
• beta (local worktree with HIPFIRE_GATED_NORM_MQ_ROTATE=0): PASS — 143.0 tok/s decode.

The clean-beta result might suggest that my uncommitted local changes are not the primary cause. The MQ4R artifact also has the expected size and digest, which makes weight corruption less likely.

Reproduction:

HIPFIRE_KV_MODE=q8 HIPFIRE_DPM_WARMUP_SECS=3 ./target/release/examples/bench_qwen35_mq4 ~/.hipfire/models/qwen3.6-35b-a3b.mq4r --prefill 8192 --prefill-runs 1 --warmup 2 --gen 8

The 8,192-token prefill completes normally, after which beta fails or stalls at === warmup (2 tokens — untimed, lets JIT settle) ===.

The same boundary also failed after a 32,768-token prefill. Short ordinary AR/MTP inference can still work, so this might be narrower than a general MQ4R failure. The reproducible trigger appears to be the long-prefill → first-single-token transition.

In this test series, git bisect points to 12a9842c, perf(gfx1100): enable gated norm MQ fusion by default, as the first bad commit.

That commit changes HIPFIRE_GATED_NORM_MQ_ROTATE from explicit opt-in to default-on. Since setting it to 0 restores successful decode in the tested beta worktree, the default-enabled fused path might be involved.

Possible next steps

• Might it make sense to keep this fusion default-off until the 8K/32K gfx1100 transition has been fixed and revalidated from the exact PR head?
• Could we confirm whether the recorded gfx1100 headline runs had HIPFIRE_GATED_NORM_MQ_ROTATE=0 set in their environment?
• Would it be useful to include the reproduction above in the final gfx1100 release hardware matrix?

I would be happy to rerun any additional configuration that might help narrow this down further.

[ghazni101]

mq4r vs mq3p VRAM A/B on gfx1100 — build flags, results, and repro

Ran an isolated A/B measuring load-time VRAM for qwen3.6-35b-a3b.mq4r vs qwen3.6-35b-a3b.mq3p on a single RX 7900 XTX (gfx1100).

Environment

• GPU: RX 7900 XTX (gfx1100), 25.75 GB VRAM
• Host ROCm 7.14; container base rocm/dev-ubuntu-24.04:7.2.4 with host ROCm overmounted at /opt/rocm/core-7.14
• Isolated Docker container (no ~/.hipfire/models.json, no per-model config overrides) so nothing overrides context/KV

Build

cargo build --release --locked --features deltanet --example daemon -p hipfire-runtime

(deltanet is required; the bench/daemon path needs it.)

Load config (identical for both models)

• MTP: off
• KV cache mode: q8
• context: 8192

VRAM measurement
Read from kernel sysfs, which is reliable through the container:

cat /sys/class/drm/card1/device/mem_info_vram_used

Peak taken as the max over rapid repeated reads during load through warm-up.

Results (peak VRAM at warm-up, MTP off, kv=q8)

model	peak VRAM	notes
mq4r	23.79 GB	uniform MQ4G256, 18.68 GB file, ~36.8B params
mq3p	~19.24 GB	mixed MQ4G256+MQ6G256+HFQ8G256+F8E4M3, 17.23 GB file, ~24.1B params
Δ	4.55 GB	of which 1.46 GB is the weight file-size difference

Per-allocation ledger (via an HIPMALLOC_LOG-gated trace build) attributed the non-weight portion to forward/DeltaNet scratch that mq4r's uniform-4bit decode path allocates and holds resident (Qwen35Scratch, speculative.rs:1008-1064); the large KV buffers were identical between the two models. FP16 shadow did not fire on gfx1100 (FP16_SHADOW_CALLED=0 for both; HIPFIRE_ROCBLAS_OFF=1 changed VRAM by 0 bytes).

Reproduce

1. Build the daemon with the flags above.
2. Run in a clean container with GPU passthrough and host ROCm overmounted:

docker run -d --name hipfire-ab --privileged \
  --device /dev/kfd --device /dev/dri \
  -v /path/to/models:/models:ro \
  -v /opt/rocm/core-7.14:/opt/rocm/core-7.14 \
  -e HIP_PATH=/opt/rocm/core-7.14 \
  -e LD_LIBRARY_PATH=/opt/rocm/core-7.14/lib:/opt/rocm/core-7.14/lib64 \
  --entrypoint /bin/sh <image> -c "while true; do sleep 3600; done"

3. Drive the daemon over a FIFO, loading each model with {"type":"load","model":"...","params":{"max_seq":8192,"kv_mode":"q8","mtp_mode":"off"}}, and poll /sys/class/drm/card1/device/mem_info_vram_used for the peak until the debug log prints warm-up complete.

[fivetide]

Replay system integration audit

Investigated whether the Redline replay system (HIPFIRE_REPLAY_BACKEND=auto) has an effect on daemon-based inference, using DeepSeek V4 Flash MTP spec-decode on gfx1151 (RYZEN AI MAX+ 395 / Radeon 8060S, ROCm 7.2).

Current state

The replay controller is completely inert in the daemon's inference path:

HIPFIRE_REPLAY_BACKEND	Result
hip (default/unset)	Works. ReplayController sits on Gpu, is_recording() returns false, no effect.
auto	Hangs — daemon gets stuck in generate, 900s timeout.
shadow	Hangs — same as auto.

Root cause

The replay state machine starts at Armed and needs begin_auto_capture_if_armed() to transition Armed → RecordingWarmup → Captured → Ready. That lifecycle function is never called from any daemon per-request path. Even if it were, the actual dispatch from every arch crate (deepseek4, qwen35, minimax, llama, etc.) goes through crate::scratch::launch_maybe_blob or raw hip.launch_kernel — neither path checks the replay controller.

The replay system was wired into the old launch_maybe_blob_bound dispatch wrapper (since refactored into crate::scratch::launch_maybe_blob on master), but the central dispatch point no longer goes through the replay recording gate either.

Performance impact: zero

Measured DeepSeek V4 MTP on both branches (clean builds, post-warmup, 3 runs each):

Config	Master (tok/s)	Beta (tok/s)	Delta
AR code	13.93	14.13	+1.4% (noise)
MTP K=2 code	21.29	21.23	-0.3% (noise)
MTP K=3 code	21.66	21.70	+0.2% (noise)

The 36 beta commits (~23K net lines) — including the Redline crates, gfx1151 i8-MMQ prefill path, attention tile tuning, and additional kernels — produce no measurable performance delta for DSv4 daemon-based spec-decode inference on this hardware. The i8-MMQ and tile32 gating conditions (large-batch prefill only) never trigger during MTP decode verify (batch_size ∈ {2,3}).

Work needed to activate replay for all model families

1. Centralize dispatch: Every arch crate's forward pass must route to a replay-aware launch point. Currently each calls crate::scratch::launch_maybe_blob or hip.launch_kernel directly, bypassing the replay wrapper.

2. Lifecycle hooks per request: begin_auto_capture_if_armed() must fire at the start of every forward pass, and the capture/replay cycle must complete within a single generate request. This requires the replay controller to be aware of each architecture's generate-loop boundaries.

3. Backend availability: Even once capture works, actual PM4 replay requires redline-rocr's AQL/PM4 packet construction and direct KMD access (device file handles, queue setup, fence sync). That path is currently only validated in standalone Redline examples, not in the daemon's multi-request lifecycle.

This is not a beta-vs-master regression — the infrastructure was added but the integration into the production inference path across arch families was not completed. The PR description correctly lists Redline as an in-tree substrate; the notes above describe the remaining gap for daemon-side activation.

[Kaden-Schutt]

@fivetide — thank you for digging into this. Your gfx1151 result found a real integration gap: HIPFIRE_REPLAY_BACKEND=auto/shadow is not currently a generic “turn PM4 on for any model” switch, and an unsupported route should fail closed or refuse early rather than sit for 900 seconds.

One scope correction, because it points toward a useful contribution: ReplayController is wired into the Qwen plain-AR adapter, but it is not wired into DeepSeek V4’s model boundary. Your test combined two currently unadmitted dimensions—DeepSeek V4 and MTP/speculative decode—so it could not exercise the narrow retained route behind the Qwen MQ4R result. MTP proposal/reseed/verify is deliberately outside the current immutable single-token tape contract.

There is now a better non-Qwen reference than the Qwen adapter: origin/lfm-redline@d335cc993. It proves that Redline can be integrated into a second model architecture rather than being Qwen-specific. The admitted fixture is still deliberately narrow—LFM2.5-350M MQ4, gfx1201, Q8 KV, single-GPU plain AR—but it produced a complete 161-dispatch / 15-kernel tape, one retained PM4-IB, bit-exact model-state shadowing, and observed product replay with no fallback.

Useful source landmarks:

• LFM admission and execution-mode boundary
• LFM capture/route/finalize lifecycle
• LFM exact-fixture negative admission gate
• Canonical model/architecture porting recipe
• Copyable new-route checklist

Please use that checkpoint as a reference, not a wholesale cherry-pick; it is a large branch checkpoint.

If you want to help with DeepSeek, the clean first contribution would be a DeepSeek plain-AR capture boundary on gfx1151, with no automatic-default or speedup claim yet:

1. Freeze one exact DeepSeek artifact/digest, gfx1151, single GPU, Q8 KV, graph off, and MTP/spec off.
2. Add a DeepSeek-owned execution-mode/eligibility predicate like LFM’s, with explicit negatives for prefill, batching, HipGraph, MTP proposal/reseed/verify, reset, model swap, and non-sequential positions.
3. Use DeepSeek’s existing decode_step / decode_step_body seam: stage token/position inputs outside the retained body, set one-forward eligibility, arm capture, route AQL/PM4, and finalize/prepare after the ordinary-HIP warmup.
4. Preserve host-side state on replay. In particular, decode_step_body currently advances state.n_tokens; a retained replay bypasses that Rust statement, so the adapter must advance the cursor explicitly after successful replay, as LFM does.
5. Reconcile the entire compute body: compute launches = retained launches + explicitly named external launches. Any raw in-body launch must become recorder-aware or be deliberately externalized with a state/lifetime justification.
6. Build the DeepSeek state oracle before optimizing: logits, main/indexer KV, SWA rings, compressor/indexer state, residual streams, mtp_last_hidden, cursor state, guards, and captured kernarg blobs.
7. Start single-queue and conservatively ordered. First prove stable tape identity and multi-position HIP/blob/PM4 parity with scripts/redline_daemon_harness.py; only afterward use scripts/redline_product_bench.py for same-report timed-arm route proof. A capture report and a separate timing report cannot be stitched into route proof.

A first PR that adds only the plain-AR adapter boundary, negative gates, and a stable/reconciled capture fingerprint would already be valuable. We can review that boundary and help work through the gfx1151 PM4 lowering from there. The immediate goal is not “make the env flag fast”; it is “make one exact DeepSeek forward safely lowerable and observable,” then optimize it deliberately.

[Kaden-Schutt]

@ghazni101 — thank you; your 23.79 GB observation was real and led us to the right pressure point. This is now fixed on beta in 0e5762b3c.

We reproduced with the same MQ4R artifact digest (4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428) on gfx1100 and ran a same-model, same-tip allocation A/B:

	Before	Fixed
Resident VRAM	23,756,001,280 B	19,406,700,544 B
GPU allocations	21,423	1,023
Requested allocation bytes	18,896,841,988 B	18,896,841,988 B

That recovers 4,349,300,736 bytes (~4.35 GB / 18.3%) without dropping any requested tensor payload.

The trace narrowed the non-weight delta further than the original cross-format ledger: uniform MQ4R created 20,480 separately owned expert-weight BOs (2 projections × 256 experts × 40 layers). HIP's per-BO/allocation overhead inflated physical residency. The fix packs each layer into two owning BOs—gate/up and down, 80 owners total—while preserving the existing per-expert pointer/offset, dtype, and AWQ-sidecar views, so the kernel ABI is unchanged.

Validation on the fixed binary:

• gfx1100 product: HIP 221.053 tok/s, PM4 250.248 tok/s
• exact replay: 604 launches / 22 kernels, stable sequence and bit-exact output
• real serve_harness.py session: 8/8 turns stopped normally, 0 runaway/empty/attractor turns; average decode 188.8 tok/s, final turn 154.7 tok/s at 18,186 context
• MQ3P fallback remained on the old loader path and its resident VRAM changed by only 2 KB
• cargo test -p hipfire-arch-qwen35 --features deltanet --lib: 147/147 passed

The optimization is deliberately gfx11-only. An initial gfx12 A/B exposed a HipGraph regression from the large shared BOs, so gfx12 retains its prior allocation behavior rather than admitting that regression.

Please rerun your exact container recipe against beta@0e5762b3c. On the RX 7900 XTX, the expected post-load residency is approximately 19.41 GB. If your transient warm-up peak materially exceeds that, please post the peak and trace and we can compare it directly.

[HUSRCF]

Follow-up: gfx1100 long-prefill to first-decode failure isolated

Thank you for the additional investigation and for fixing the separate MQ4R allocation-count issue in 0e5762b3. It appears that there were two independent problems: the multi-BO residency overhead explained the ~4.35 GB VRAM delta, while the long-prefill-to-first-decode failure came from a much smaller but correctness-critical scratch-buffer overflow.

I isolated the latter failure to Qwen35Scratch::x_rot:

• Qwen3.6-35B-A3B has dim=2048, hidden_dim=0, and DeltaNet v_dim=32*128=4096.
• x_rot was allocated as max(dim, hidden_dim), giving 2048 F32 elements.
• gated_norm_mq_rotate_gfx1100 writes all 4096 F32 output elements.
• The fused launch therefore wrote 2048 F32 elements, or exactly 8192 bytes, beyond the allocation.

This explains why setting HIPFIRE_GATED_NORM_MQ_ROTATE=0 avoided the failure: it bypassed the kernel writing the undersized rotation buffer. The default-on change in 12a9842c exposed the existing allocation mismatch rather than introducing a weight-file problem.

I opened PR #538 against beta. It:

1. Sizes x_rot as max(dim, hidden_dim, v_dim).
2. Checks the exact fused shape and all tensor capacities before launch.
3. Adds a regression test for the Qwen3.6-35B-A3B dimensions.

The memory cost is only 8 KiB per scratch instance, so it is unrelated to the multi-gigabyte residency issue fixed in 0e5762b3. GitHub currently reports #538 as cleanly mergeable against that beta tip.

Validation used the same MQ4R artifact digest reported earlier:

4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428

• The old allocation with the new launch guard fails cleanly and reports x_rot=2048, required=4096, rather than issuing an illegal write.
• The fixed default-on fused path completes the 8192-token prefill, 2-token warmup, and 8-token decode without HIP 700 or a stall.
• The final regression run measured 145.9 tok/s decode.
• Fusion on/off produced identical token IDs in the diagnostic parity run.
• test_kernels: 16/16 passed on Radeon Pro W7900 / gfx1100.
• Workspace build, library tests, and no-GPU CI passed.

There is no performance claim from the single-run on/off timing; the important result is restoring correctness at the reproducible transition boundary while retaining the default-enabled fusion. This PR intentionally does not change the fused-kernel algorithm or dispatch policy: it only corrects scratch capacity and adds pre-launch validation. The resulting VRAM increase is just 8 KiB per scratch instance.

[ghazni101]

Verification: commit 0e5762b resolves the ~4 GB mq4r vs mq3p VRAM gap

Tested on gfx1100 (RX 7900 XTX, 24 GB VRAM, ROCm core-7.14, HIP 7.14).

Build

git fetch origin 0e5762b3c92d0f60f2dd454b8018d0b8dadfebc9
cargo build --release --locked --features deltanet --example daemon -p hipfire-runtime

Binary confirmed: 22 MB ELF, PackedExpertOwners symbols present in nm.

Methodology

• Fresh Docker container per load (isolated, no cross-contamination)
• Daemon driven via FIFO: {"type":"load","model":"...","params":{"max_seq":512,"kv_mode":"q8","mtp_mode":"off"}}
• VRAM read from host: /sys/class/drm/card1/device/mem_info_vram_used (5 consecutive stable samples)
• MTP sidecar moved to /tmp for the clean baseline; verified no speculator/spec_step/MTP speculator enabled in daemon debug for any run
• The MTP head loaded message (weight load only) is absent when the sidecar is removed

Results

With MTP sidecar loaded (446 MB on disk / ~582 MB in VRAM)

Model	Peak VRAM
mq4r	19,838 MB
mq3p	19,517 MB
Δ	321 MB

Without MTP sidecar (moved to /tmp, zero MTP lines in log)

Model	Peak VRAM
mq4r	19,256 MB
mq3p	18,934 MB
Δ	322 MB

Both show the same ~320 MB gap regardless of MTP (MTP adds ~582 MB equally to both models) and regardless of KV cache size (tested both max_seq=512 and max_seq=8192 — identical delta).

File size vs VRAM delta

Metric	mq4r	mq3p	Δ
On-disk file	18.70 GB	17.24 GB	1.46 GB
Post-load VRAM (no MTP)	19,256 MB	18,934 MB	322 MB

The 1.46 GB file-size delta does not carry through to VRAM because the two quant formats (MQ4 vs MQ3 group schemes, e.g. quant_type=13 vs quant_type=3 on lm_head.weight) have different on-disk compression ratios but decompress to the same fp16 runtime layout for the weight tensors. The ~320 MB VRAM residual is the remaining real difference in weight footprint + scratch after the packed-expert fix.

Verdict

The original gap of ~4,200 MB (measured July 17 on the beta build) is now ~320 MB under commit 0e5762b. The fix for the per-expert 512-BO allocation overhead in PackedExpertOwners works correctly on this gfx1100 card.

[Kaden-Schutt]

Rust-only control plane landed on beta in 2a960e19017b3e108b168d653510f9be349397ee.

What changed:

• Removed the tracked TypeScript/Bun CLI surface and replaced it with native hipfire-cli, hipfire-client, hipfire-config, and hipfire-registry crates.
• Moved user configuration and per-model overrides to typed TOML with legacy JSON/env migration.
• Moved the OpenAI-compatible HTTP/SSE service, registry handling, TUI integration, installers, container packaging, and serve_harness.py launch path onto the Rust control plane.
• The committed beta tree contains no tracked .ts/.tsx, package.json, tsconfig.json, or Bun lockfiles.

Validation:

• HIPFIRE_NO_REGISTRY_FETCH=1 bash scripts/no-gpu-ci.sh passed: workspace/example checks, 87 rdna-compute tests, 9 Qwen MoE-prefill tests, the new Rust control-plane crate suites, and 381 Python CPU tests.
• A fresh release build of hipfire-cli passed.
• Real MQ4R registry/TOML resolution through serve_harness.py --show-config produced the intended q8 / xhigh / max-4096 configuration.
• A protocol-faithful synthetic daemon completed a 5/5 serve_harness.py battery through the full native CLI -> HTTP/SSE -> JSONL daemon path, including prewarm, health, streaming, usage/timings, and cleanup.
• redline_daemon_harness.py and redline_product_bench.py continue to speak directly to the daemon rather than the CLI; their unit tests and complete synthetic control flows passed.

Scope note: this validates the control-plane migration and script wiring. It is not a fresh physical-GPU or retained-PM4 certification, and this commit does not change Redline kernels or the daemon protocol. The Nix expressions were source-reviewed, but Nix tooling was not installed locally for parser evaluation.

[ghazni101]

Performance and 200K context on gfx1100 (beta@2a960e1)

Re-ran the headline Qwen3.6-35B-A3B MQ4R path against the latest beta HEAD.

Performance — TG128 after proper thermal soak

Measured through the daemon FIFO protocol after 5x pre-heat generations (GPU at thermal steady state). The daemon's decode_tok_s field is the clean decode metric:

decode_tok_s: 212.3  (TG128, ~20-tok prompt, 5 gen warmup)
prefill_tok_s: 900.6
TTFT: 22.2 ms

Decode is flat at ~211-213 tok/s across all tested generation lengths (64-8192, sigma < 0.5 tok/s) and prompt lengths (16-1024). First generation after a cold GPU is 5-15% slower — heat soak matters for reproducible numbers.

Your 253.3 tok/s reproduces on this card at 212.3 tok/s (16% gap). GPU hits full boost (3139 MHz core, 1249 MHz memory) at 306W draw vs the 339W cap — the remaining gap is card-level power/clock variation, not software. Tested both ROCm 7.14 and 7.2, HIPFIRE_GRAPH=1, and COMPUTE power profile — none moved the needle.

200K context

The daemon loaded and served with max_seq=200000 at q8 KV:

• VRAM at load: 22.88 GiB (95.4% of 23.98 GiB)
• Decode at 200K: 210.7 tok/s (statistically identical to short-context)
• Prefill cost at 200K increased marginally (TTFT ~47 ms vs ~22 ms at short ctx)

200K context fits on 24 GiB with ~1 GiB headroom for generation scratch. Decode speed does not degrade with the larger KV allocation — only attention prefill costs increase.

[Kaden-Schutt]

@ghazni101 — thanks, this result was useful. We reproduced your ~212 tok/s band as the ordinary HIP/HipGraph arm. The 253.3 tok/s headline is the retained-PM4 route; HIPFIRE_GRAPH=1 enables HipGraph, but it does not select or prove Redline PM4.

I have fast-forwarded beta to 78b75f04a. This head adds a route-proof-capable product benchmark and fixes the typed replay selection reaching the product arm.

For a matched TG128 reproduction, please use the benchmark below rather than hand-driving the daemon FIFO:

git fetch origin beta
git switch beta
git pull --ff-only origin beta

cargo build --release --locked --features deltanet \
  -p hipfire-runtime --example daemon

sha256sum ~/.hipfire/models/qwen3.6-35b-a3b.mq4r
# expected:
# 4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428

python3 scripts/redline_product_bench.py \
  --model ~/.hipfire/models/qwen3.6-35b-a3b.mq4r \
  --daemon target/release/examples/daemon \
  --context 128 --iterations 128 \
  --warmups 10 --warmup-iterations 32 --runs 10 \
  --transport pm4 --kv-mode q8 --max-seq 2048 \
  --work-dir .redline-work/gfx1100-pm4 \
  --out .redline-work/gfx1100-pm4/report.json

jq '{valid, hip: .hip.tok_s, pm4: .auto.tok_s, speedup, route: .auto.route_proof}' \
  .redline-work/gfx1100-pm4/report.json

Leave the stationarity defaults intact. A valid result must have:

• top-level valid: true;
• .hip.route_proof.retained_rows == 0;
• .auto.route_proof.valid == true;
• .auto.route_proof.retained_rows == 10;
• .auto.route_proof.errors == [].

Our same-binary gfx1100 rerun measured HIP 218.32 tok/s and retained PM4 251.57 tok/s (251.17–251.70), with all ten timed rows positively observing replay. The retained tape was 604 launches / 22 kernels, sequence hash be77b87a0f994622.

For ordinary hipfire serve, use the typed config surface instead of adding more environment variables:

hipfire config set hardware.devices 0
hipfire config set memory.kv_cache q8
hipfire config set speculation.mode off
hipfire config set replay.backend redline
hipfire config set replay.transport pm4
hipfire config list

hardware.devices is the physical ROCr selector; Hipfire derives and synchronizes the matching logical HIP selector. If the benchmark reports a fallback, state != "ready", or no retained rows, please post the JSON report rather than comparing its throughput to the PM4 headline.

[taniguchi-taku-softm]

also tested.
did i mistake anything?

git fetch origin beta
git switch beta
git pull --ff-only origin beta

cargo build --release --locked --features deltanet \
  -p hipfire-runtime --example daemon

sha256sum ~/.hipfire/models/qwen3.6-35b-a3b.mq4r
# expected:
# 4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428

python3 scripts/redline_product_bench.py \
  --model ~/.hipfire/models/qwen3.6-35b-a3b.mq4r \
  --daemon target/release/examples/daemon \
  --context 128 --iterations 128 \
  --warmups 10 --warmup-iterations 32 --runs 10 \
  --transport pm4 --kv-mode q8 --max-seq 2048 \
  --work-dir .redline-work/gfx1201-pm4 \
  --out .redline-work/gfx1201-pm4/report.json

jq '{valid, hip: .hip.tok_s, pm4: .auto.tok_s, speedup, route: .auto.route_proof}' \
  .redline-work/gfx1201-pm4/report.json

...

4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428  /opt/zen/.hipfire/models/qwen3.6-35b-a3b.mq4r
hip: warming caches... took 2.45s
hip: stationary after 20 full-tg rows (15.81s, median=173.363, slope=-0.0078%/row, spread=0.221%)
hip: measured median=173.493 tok/s valid=True
hip: timed route proof valid=True retained_rows=0 positions=[] lifecycle_valid=True
auto: warming caches... took 3.00s
auto: stationary after 39 full-tg rows (37.16s, median=141.778, slope=-0.0003%/row, spread=0.555%)
auto: measured median=141.683 tok/s valid=True
auto: timed route proof valid=True retained_rows=10 positions=[128, 255] lifecycle_valid=True
hip=173.493 tok/s auto=141.683 tok/s speedup=0.81665 valid=True report=.redline-work/gfx1201-pm4/report.json
{
  "valid": true,
  "hip": {
    "min": 173.26213295632,
    "median": 173.49262094976075,
    "max": 173.6050426579327
  },
  "pm4": {
    "min": 141.52877635006118,
    "median": 141.68284429407555,
    "max": 141.8537463981145
  },
  "speedup": 0.8166505498530884,
  "route": {
    "valid": true,
    "backend": "auto",
    "transport": "pm4",
    "rows": 10,
    "require_complete_replay": true,
    "retained_rows": 10,
    "observed_positions": [
      128,
      255
    ],
    "prepared_identities": [
      [
        733,
        1,
        2,
        15960
      ]
    ],
    "sequences": [
      [
        733,
        23,
        "3318ffca3daf2338"
      ]
    ],
    "errors": []
  }
}

[ghazni101]

@Kaden-Schutt — reran the matched TG128 reproduction with the route-proof benchmark as requested. Result: valid, PM4 route positively proven on gfx1100.

Environment

• GPU: AMD Radeon RX 7900 XTX (gfx1100), 24 GiB
• ROCm 7.2.4 (containerized: rocm/dev-ubuntu-24.04:7.2.4-complete base), HIP 7.2
• hipfire beta @ 78b75f04a, built exactly as specified:
  cargo build --release --locked --features deltanet -p hipfire-runtime --example daemon
• Model: qwen3.6-35b-a3b.mq4r, sha256 verified 4685c140c46b1a6f31a0fd9053bf09d5faf1d2529d715b84794249b66cde0428
• Bench: scripts/redline_product_bench.py --context 128 --iterations 128 --warmups 10 --warmup-iterations 32 --runs 10 --transport pm4 --kv-mode q8 --max-seq 2048

Result

Arm	tok/s (min / median / max)	route proof
HIP (ordinary)	223.08 / 223.22 / 223.41	retained_rows=0, valid
PM4 (retained)	245.87 / 247.59 / 247.79	retained_rows=10, valid

• Speedup: 1.109×
• Top-level valid: true; .auto.route_proof.valid == true; .auto.route_proof.errors == []; all ten timed rows positively observed replay (observed_positions [128, 255])
• Retained tape: 604 launches / 22 kernels, prepared identity [604, 1, 2, 16832], sequence hash 43754a60ca25f47c

This matches your same-binary gfx1100 rerun (HIP 218.32 / PM4 251.57) in direction and magnitude — ~10% retained-PM4 gain over the ordinary HIP arm.

One deviation, disclosed

With everything at defaults, the run aborted twice with auto failed to become stationary after 60 full-tg rows. The PM4 arm's rolling window was already inside your gates near the cap (run 2: slope −0.021 %/row vs 0.05 limit, spread 0.598 % vs 1.0 limit, stable: true at rows 51–60) — it just didn't accumulate the 10 confirmation rows before --settle-max-runs 60 cut it off. HIP settles in ~20–30 rows here; PM4 needs ~57.

So the passing run used --settle-min-runs 20 --settle-max-runs 120 — i.e. a larger measurement budget only. The stationarity criteria (slope/spread/median-drift gates and confirmation count) were left at defaults, per your instruction. Final PM4 settle: stationary after 57 rows, slope −0.029 %/row, spread 0.800 %.

You may want to consider whether the default 60-row cap is tight for gfx1100 cold starts; happy to rerun with any settings you prefer.

[Kaden-Schutt]

@taniguchi-taku-softm — no, you did not make a mistake. Your run was correct, and it found a real gfx12 defect. Thank you for posting the full report rather than just the speedup number.

Your tape was already the right one. Comparing your report against the now-sealed gfx1201 fixture:

	your run @ 78b75f04a	sealed fixture @ 319905cb4
dispatches	733	733
unique kernels	23	23
sequence hash	3318ffca3daf2338	3318ffca3daf2338
command dwords	15,960	20,511
HIP median	173.493 tok/s	174.670 tok/s
PM4 median	141.683 tok/s	202.460 tok/s

Identical dispatch count, identical kernel count, identical sequence hash, and HIP arms within 0.7% — so you captured exactly the intended route on comparable hardware. The only structural difference was the command-dword count, and that was the bug.

Root cause: the retained PM4 path was caching register state that gfx12 requires re-emitted per dispatch. On gfx11 that state is invariant across replay and caching it is a win; on gfx12 part of it is compiler-dependent and must be re-emitted, so the retained tape was under-emitting and the replay arm lost to ordinary HIP. Fixed in 319905cb4 by defaulting retained PM4 to static register emission while still caching queue-global invariants. That is what moves 15,960 → 20,511 dwords and 141.68 → 202.46 tok/s.

Your report also drove two follow-ups:

• The measured route is now sealed in registry/redline-golden-v1.json with the gfx1201 acceptance floor at 197.0 tok/s and 1.10x, so this class of regression fails closed rather than depending on whoever happens to re-run the bench.
• The stationarity ceiling moved from 60 to 120 rows. Your PM4 arm settled in 39, but @ghazni101's gfx1100 PM4 arm needed 57 and was being cut off at the old default.

To re-run against the sealed fixture:

git fetch origin beta && git switch beta && git pull --ff-only origin beta
python3 scripts/golden-redline.py --arch gfx1201

That verifies the registry card, sampling profile, model size and SHA-256, builds the release daemon if absent, runs the TG128 HIP-versus-retained-PM4 product benchmark with the sealed PM4 policy and stationarity parameters, and requires the exact 733-launch / 23-kernel / 3318ffca3daf2338 tape plus positive timed-arm route proof before it evaluates throughput at all. Add --device N if gfx1201 is not physical device 0, and --dry-run first if you want to see the exact command without building, hashing, or touching the GPU.

Expected: PM4 >= 197.0 tok/s at >= 1.10x. If it lands under that with a valid route proof, please post the JSON report again — the fixture is recorded evidence, not a timeless guarantee, and a second board disagreeing with it is exactly what I want to know about.

See docs/GOLDEN-REDLINE.md for the full pass criteria.