Skip to content

feat(utils): add once-per-message logger methods - #858

Merged
jason-fxz merged 1 commit into
mainfrom
fix/kernel-select-log-once
Nov 3, 2026
Merged

jason-fxz merged 1 commit into
mainfrom
fix/kernel-select-log-once

Conversation

@jason-fxz

Copy link
Copy Markdown
Collaborator

Summary

Loading a quantized checkpoint prints the kernel choice once per layer: select_kernel runs for every layer's QuantMethod and logs each time. nvidia/Qwen3.6-35B-A3B-NVFP4 (ModelOpt mixed precision, W4A16_NVFP4 experts) prints 121 lines with only 2 distinct messages:

kernel triton selected; skipped w4a4: the experts declare no static activation scale (W4A16 checkpoint)
kernel triton selected; skipped b12x: the weight declares no static activation scale (W4A16 checkpoint); cublas: the weight declares no static activation scale (W4A16 checkpoint)

This adds debug_once, info_once and warning_once to the loggers init_logger returns, following vLLM's logger.info_once (default scope "local"): rank 0 only, once per distinct (logger, level, msg, args) in the process. The seen set is module-level because init_logger rebuilds a logger's methods on every call for the same name. Args must hash by value, as with vLLM's lru_cache key.

Moved onto it:

  • layers/quantization/method.py: select_kernel.
  • layers/quantization/moe/nvfp4.py: the "installed but unusable" warnings of the marlin and b12x symbol checks, which unusable_reason runs for every MoE layer.
  • layers/quantization/linear/nvfp4.py: _warn_part_globals (functools.cache + warning_rank0) removed.
  • models/gguf/mmproj.py: _warn_inline_vision (functools.cache per model path) removed.
  • models/minimax_m3/config.py: _log_mode_once and _LOGGED_MODES removed.
  • models/qwen3_vl/vision.py: the _warned_rope_fallback flag removed.

These lines now log on rank 0 only; before, the minimax, mmproj, qwen3_vl and MoE symbol-check lines logged on every rank.

Left as they are, since they suppress differently: scheduler.py _warned_cut_image (first warning only, the text varies per image), server/request_logger.py _warn_once (one flag for every message), moe/offload_cache.py hit-d2d flag (re-arms after a rebuild).

Tests

  • 1x RTX PRO 6000 Blackwell Server Edition (sm_120, 96 GB), driver 595.91.07, torch 2.14.1+cu130.
  • ft serve --model nvidia/Qwen3.6-35B-A3B-NVFP4 --port 30112 (local copy): 121 kernel ... selected lines on main 9cacd08, 2 on this branch.
  • pytest -m "not slow" tests/models/test_quant_config.py tests/models/test_minimax_m3.py tests/models/test_minimax_m3_vision.py tests/models/test_qwen3_vl.py tests/models/test_qwen3_vl_vision.py tests/models/qwen4_exp/test_skeleton.py tests/models/test_gemma4_gguf_rope.py tests/models/test_models_registry.py tests/kernels/test_nvfp4_w4a4_linear.py tests/kernels/test_e4m3_compat.py tests/moe: 393 passed, 50 skipped, 8 failed; the same 8 fail on main 9cacd08 (7 in test_quant_config.py from local checkpoints whose quant dialect differs from the HF repo the case names, or index-only MiniMax-M3-NVFP4 snapshots; 1 in tests/moe/test_nvfp4_backends.py::test_w4a4_prefill_matches_nvfp4_activation_reference[512-1024-swiglu_clamp-256], which passed in an earlier run on both).
  • No new test.

debug_once / info_once / warning_once log on rank 0, once per distinct message and args. select_kernel, which logged its choice once per layer, and the hand-rolled log-once guards now use them.
@jason-fxz
jason-fxz merged commit 33cbb3c into main Nov 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant