Track 3: MuonH power-0.4 schedule reaches 3.28 at 3150 steps

[mangocrazz]

Summary

This improves the accepted MuonH per-optimizer result #37 from 3250 to 3150 steps.

• MuonH updates 0–99: linear warmup from 1e-5 to 0.03
• updates 100–199: hold at 0.03
• updates 200–3200: floor + (peak-floor) * (1-progress**0.4)
• absolute floor: 1e-5
• auxiliary AdamW settings and schedule are unchanged from Has anyone tried the Cautious Optimizer? #37
• model architecture, data, global batch, and single forward/backward per update are unchanged

The exact run implementation also uses float32 RMS normalization, as disclosed in the result README and reproduced in the self-contained script. This implementation is already present in accepted Track-3 entries.

Statistical result

Seed 1 was used during the schedule sweep, so the primary claim excludes it and uses the 11 held-out runs (seeds 0 and 2–11).

Step	Mean val loss (n=11)	(3.28 - mean) * sqrt(11)
3125	3.27950818	0.00163118
3150	3.27778727	0.00733879
3175	3.27654727	0.01145140

The required threshold is 0.004, so 3150 passes. For completeness, all 12 seeds including seed 1 give mean 3.27783333 and statistic 0.00750555.

All runs continued to step 3250; 3150 is a common validation point, not per-run early stopping.

Comparisons

• Versus accepted Has anyone tried the Cautious Optimizer? #37 (3250 steps, 3.2786, n=10), the README step-adjusted pairwise statistic is 0.01216.
• Versus open Track 3 (Per-optimizer SOTA): MuonH with minus-sqrt LR schedule — 3175 steps (n=9) #343, this reaches the target 25 steps earlier, but the pairwise statistic is only 0.00191. I therefore do not claim a statistically significant head-to-head win over Track 3 (Per-optimizer SOTA): MuonH with minus-sqrt LR schedule — 3175 steps (n=9) #343.

Reproduction

The result directory includes:

• a self-contained reproduction script with no external optimizer dependency;
• exact raw logs for seeds 0–11;
• the complete per-seed table and both held-out/all-run calculations;
• the initial and resumed logs for seed 5, which was preempted after a committed checkpoint.

The submitted runs used 16 GPUs with the standard 524,288-token global batch.