Skip to content

fix(v4): report expert-matmul phase in serve telemetry — web dashboard (#890) - #1005

Merged
JustVugg merged 1 commit into
devfrom
fix/v4-phase-telemetry
Aug 14, 2026
Merged

fix(v4): report expert-matmul phase in serve telemetry — web dashboard (#890)#1005
JustVugg merged 1 commit into
devfrom
fix/v4-phase-telemetry

Conversation

@JustVugg

Copy link
Copy Markdown
Owner

#890: DeepSeek V4's "Turn wall time by phase" is 100% "other" — I/O and matmul both 0%.

Root cause (two layers): v4_prof_emit hardcoded every phase but disk to 0.000, with a comment claiming the runtime "has no per-phase instrumentation beyond disk-read time". That's not true — and it misses the real problem: on a box with enough RAM the model sits warm in page cache, so disk_sec ≈ 0 as well, and the compute phase (the actual time when the disk is warm) was never measured at all. Everything folded into "other".

Fix follows GLM's pattern (its m->t_emm timer): a counter on the shared store instance — not a per-unit global — so it works across the per-COLI_V4_UNIT builds without a linking problem. matmul_sec sits beside disk_sec in V4ExpertStoreState, accumulated around the expert forward in moe_token_pipeline, and read per-turn in v4_serve_one exactly like disk_sec. The dashboard is unchanged — it already parses these fields; V4 just fills them.

Verified:

  • builds clean across the unit split (the extern is declared in both the block unit that calls add and the serve unit that reads the getter — the warning I hit and fixed);
  • deepseek-v4-tiny-check token-exact → the timing hooks don't touch model output.

Honest scope:

  • Runtime confirmation of the non-zero matmul field on the live dashboard is best done on a box that can hold the 284B checkpoint (this dev box is RAM-starved as I write). The logic mirrors disk_sec, which already renders correctly, so the risk is low — @acedogblast, if you pull this, the Profiling tab should show I/O + matmul instead of all-other.
  • attention still folds into "other" (it lives in a struct the store doesn't reach); a follow-up can add it the same way.

Refs #890 — will close it once the matmul field is confirmed live.

🤖 Generated with Claude Code

#890)

The dashboard's "Turn wall time by phase" was 100% "other" for DeepSeek
V4: v4_prof_emit hardcoded every phase but disk to 0.000, with a comment
claiming "the V4 runtime has no per-phase instrumentation beyond
disk-read time". Two problems: on a box with enough RAM the model sits
warm in page cache, so disk_sec ~ 0 too and even I/O read as other; and
the compute phase — the actual time when the disk is warm — was simply
never measured.

Fix follows GLM's pattern (m->t_emm): a timer on the shared store
instance, not a per-unit global, so it works across the per-COLI_V4_UNIT
builds without a linking problem. matmul_sec sits beside disk_sec in
V4ExpertStoreState (both definitions), accumulated around the expert
forward in moe_token_pipeline via coli_v4_expert_store_add_matmul, and
read per-turn in v4_serve_one exactly like disk_sec. v4_prof_emit now
emits the real matmul field; "other" shrinks to attention + framing.

The dashboard is unchanged — it already parses these fields; V4 just
fills them instead of sending zeros.

Verified: builds clean across the unit split (the extern is declared in
both the block unit that calls add and the serve unit that reads the
getter); deepseek-v4-tiny-check stays token-exact, so the timing hooks
don't touch model output. Runtime confirmation of the non-zero matmul
field on the live dashboard is best done on a box that can hold the
284B checkpoint (this one is RAM-starved right now); the logic mirrors
disk_sec, which already renders correctly.

Follow-up (#890): attention-phase timing still folds into "other" — it
lives in a struct the store doesn't reach; a later pass can add it the
same way.

Refs #890

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JustVugg
JustVugg merged commit 90a6e1f into dev Aug 14, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant