Skip to content

Unify accuracy-mode configuration and preserve explicit clipping - #12

Merged
zrr1999 merged 6 commits into
PFCCLab:mainfrom
zrr1999:fix/glm45-reproducible-grad-norm
Sep 16, 2026
Merged

zrr1999 merged 6 commits into
PFCCLab:mainfrom
zrr1999:fix/glm45-reproducible-grad-norm

Conversation

@zrr1999

@zrr1999 zrr1999 commented Sep 15, 2026 •

Copy link
Copy Markdown

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

Use the existing use_accuracy_compatible argument for model and optimizer numerics. Delete the SFT initialization override that silently sets clip_grad to zero, and remove the separate reproducible-norm and native-unfused-AdamW arguments. DSA padding and bridge behavior follow the model architecture and shared mode. An older Megatron optimizer configuration that cannot honor this mode fails explicitly.

Requires Megatron #17; PaddleFleet #1961 removes the retired options from the GLM cases. Reference wheels must include both companion changes.

Experiment results

  • 27 focused tests pass, including all requested clipping thresholds in both modes, configuration propagation, ordinary-model padding, DSA bridge behavior, and collator construction before model_info.config is populated.
  • Final owning heads ran native Paddle and Torch entrypoints for 100 steps in formal-ci-followup-20260915-n100-r1: all 100 loss values are IEEE bit-identical, provenance passes, and both canonical safetensors checkpoints match. Each side also matches its prior accepted 100-step loss and checkpoint. GLM4.5 native EP2 10-step regression has 40/40 matching loss hashes and matches the previous baseline on both sides. Local receipts: experiments/ops/ci-followup-20260915/terminal.json and glm45-pair-r0/comparison.json. Frozen repository pins, existing completion-required items and the incomplete native-build snapshot remain unresolved; this is candidate acceptance, not complete + sealed or remote-CI acceptance.
  • Flake8, isort and YAPF pass on the changed files; the final padding change also passes the repository pre-commit hooks under CI Python 3.10.

@zrr1999 zrr1999 changed the title Preserve explicit clipping with reproducible gradient norms Unify accuracy-mode configuration and preserve explicit clipping Sep 15, 2026
@zrr1999
zrr1999 marked this pull request as ready for review September 16, 2026 02:00
@zrr1999
zrr1999 merged commit 45908be into PFCCLab:main Sep 16, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant