fix(v4): report expert-matmul phase in serve telemetry — web dashboard (#890) - #1005
Merged
Conversation
#890) The dashboard's "Turn wall time by phase" was 100% "other" for DeepSeek V4: v4_prof_emit hardcoded every phase but disk to 0.000, with a comment claiming "the V4 runtime has no per-phase instrumentation beyond disk-read time". Two problems: on a box with enough RAM the model sits warm in page cache, so disk_sec ~ 0 too and even I/O read as other; and the compute phase — the actual time when the disk is warm — was simply never measured. Fix follows GLM's pattern (m->t_emm): a timer on the shared store instance, not a per-unit global, so it works across the per-COLI_V4_UNIT builds without a linking problem. matmul_sec sits beside disk_sec in V4ExpertStoreState (both definitions), accumulated around the expert forward in moe_token_pipeline via coli_v4_expert_store_add_matmul, and read per-turn in v4_serve_one exactly like disk_sec. v4_prof_emit now emits the real matmul field; "other" shrinks to attention + framing. The dashboard is unchanged — it already parses these fields; V4 just fills them instead of sending zeros. Verified: builds clean across the unit split (the extern is declared in both the block unit that calls add and the serve unit that reads the getter); deepseek-v4-tiny-check stays token-exact, so the timing hooks don't touch model output. Runtime confirmation of the non-zero matmul field on the live dashboard is best done on a box that can hold the 284B checkpoint (this one is RAM-starved right now); the logic mirrors disk_sec, which already renders correctly. Follow-up (#890): attention-phase timing still folds into "other" — it lives in a struct the store doesn't reach; a later pass can add it the same way. Refs #890 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#890: DeepSeek V4's "Turn wall time by phase" is 100% "other" — I/O and matmul both 0%.
Root cause (two layers):
v4_prof_emithardcoded every phase but disk to0.000, with a comment claiming the runtime "has no per-phase instrumentation beyond disk-read time". That's not true — and it misses the real problem: on a box with enough RAM the model sits warm in page cache, sodisk_sec≈ 0 as well, and the compute phase (the actual time when the disk is warm) was never measured at all. Everything folded into "other".Fix follows GLM's pattern (its
m->t_emmtimer): a counter on the shared store instance — not a per-unit global — so it works across the per-COLI_V4_UNITbuilds without a linking problem.matmul_secsits besidedisk_secinV4ExpertStoreState, accumulated around the expert forward inmoe_token_pipeline, and read per-turn inv4_serve_oneexactly likedisk_sec. The dashboard is unchanged — it already parses these fields; V4 just fills them.Verified:
addand the serve unit that reads the getter — the warning I hit and fixed);deepseek-v4-tiny-checktoken-exact → the timing hooks don't touch model output.Honest scope:
disk_sec, which already renders correctly, so the risk is low — @acedogblast, if you pull this, the Profiling tab should show I/O + matmul instead of all-other.Refs #890 — will close it once the matmul field is confirmed live.
🤖 Generated with Claude Code