Track 1: terminal-blended readout TailEMA (1.261 min, -0.895%)

[deepmatmul]

Summary

• Maintain FP32 TailEMA shards for lm_head and the split embedding readout, then fuse a 65% terminal blend into the existing compiled Adam update and ordinary all-gather.
• Initialize MLP c_proj at 0.5 * d_model**-0.5 instead of zero.
• Reduce the schedule from 1300 to 1288 total steps.

This is based on current training code at 003ff3e, which already includes merged PR #317. The newer bc1b58e commit is a README-only leaderboard update and is included in this branch.

Same-hardware result

One dedicated 8xH100 SXM 80GB node, eight frozen balanced blocks (ABBA/BAAB), 16 runs per arm:

Arm	n	Mean time	SD	Mean loss	Loss p
current master	16	76.3385s	0.0364s	3.2776125	—
this PR	16	75.6554s	0.0365s	3.2777875	7.97e-05

• Paired geometric runtime ratio: 0.99105219 (0.8948% faster)
• 95% CI: [0.99064263, 0.99146191] (0.8538%–0.9357% faster)
• One-sided paired timing p-value: 1.378e-10
• Listed merged record: 1.266 min; candidate mean: 1.261 min

An independent frozen 10-run candidate gate also passed: mean loss 3.2781500, one-sided p = 0.003473.

Validation and evidence

• 32 final paired full logs, 10 independent accuracy logs, hardware/hash records, frozen runner, and chronological disclosures are under records/track_1_short/2026-07-27_CurrentMasterTailEMA/.
• Final train_gpt.py SHA-256: ee9586a5579a94d135ad77a826538553566ecd398b9ca3c49729ca3107969063.
• The analyzer reproduces all values in RESULTS.txt.
• A full-shard compiled H100 numerical oracle passed source auditing against the eager semantic reference.
• No training/validation data changes and no extra torch._inductor.config or torch.compile flags.

Tail averaging is inspired by Jesse Clark and the Track 3 tail-reference/TailEMA work; the record README includes detailed attribution and implementation notes.

[ClassicLarry]

Cool. Could you please rebase on main, and simplify the PR to just a couple logs and train.py?