[Fix] Keep SignSGD momentum buffer across steps

[11NOel11]

Problem (Why?)

SignSGD.init_group() is called from step() on every optimizer step, and when momentum > 0 it
reassigns state['momentum_buffer'] = torch.zeros_like(p) unconditionally. The buffer is zeroed
before every update, so it never carries state from one step to the next.

Since the update is sign(buf) and buf collapses to (1 - momentum) * grad, the positive scale
factor is discarded by sign. Signum therefore behaves as plain SignSGD, and momentum has no
effect on training. momentum=0.9 is the constructor default, so SignSGD(params, lr=...) is
affected.

A four-value check on a small network, 50 steps each, gives identical loss trajectories and zero
parameter difference:

momentum	final loss	max abs param diff vs momentum=0
0.0	0.7272830606	0.0
0.5	0.7272830606	0.0
0.9	0.7272830606	0.0
0.99	0.7272830606	0.0

With a constant gradient the buffer stays at (1 - momentum) * grad instead of accumulating, and at
an exactly zero gradient the optimizer makes no update where Signum would still step along the
carried moment.

Solution (What/How?)

Initialise the buffer only when it is absent. SGDW (sgd.py:183) and AccSGD (sgd.py:71) in
the same module already use the if len(state) == 0 pattern.

Other changes (bug fixes, small refactors)

Added test_sign_sgd_preserves_momentum_buffer, parametrised over both foreach paths. After two
steps with gradients 1.0 then -0.1 and momentum=0.9, the buffer should be
0.9 * 0.1 + 0.1 * (-0.1) = 0.08. On main it is -0.01.

Notes

While looking at this I also noticed that weight_decay and weight_decouple are validated and
stored in defaults but never applied in either SignSGD step path. I have left that out of this
PR because fixing it changes behaviour for anyone currently passing a non-zero weight_decay, which
seems like your call rather than mine. I opened a separate issue for it and am happy to send a patch
if you want one.

SGDSaI has the same unguarded initialisation, though it may be intentional there since its state
is set up during warmup. I have not touched it.

Checklist

• Make sure to run just format before commit
• My code adheres to the style guidelines of this project (just check shows no errors)
• Both new and existing unit tests pass successfully on my local environment by running just test
• I have made the necessary changes to the documentation

ruff check passes on both changed files. I did not run pyright, so please flag anything it
catches. The docs box is unticked because I do not think this needs a docs change, but say if it
does.

Test suite before the change: 2486 passed, 2 failed, 433 skipped. After: 2488 passed, 2 failed, 433
skipped. The two failures are test_complex_optimizers[FTRL_...] and test_parse_version; both
also fail on an unmodified checkout of main and are unrelated to this change.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (b452a0f) to head (ea69ccb).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #516   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          128       128           
  Lines        12442     12442           
=========================================
  Hits         12442     12442           

Flag	Coverage Δ
3.10	100.00% <100.00%> (ø)
3.11	100.00% <100.00%> (ø)
3.12	100.00% <100.00%> (ø)
3.13	100.00% <100.00%> (ø)
3.14	100.00% <100.00%> (ø)
3.8	99.98% <100.00%> (ø)
3.9	99.98% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kozistr]

Hi! Thanks for your contribution. looks good to me!