[iakraptis]

What this changes

Replaces policy/static-policy.json with a v2 policy trained by League PPO plus a Magnetic Mirror Descent (MMD) term. It beats the current champion (iteration 2995) 55.98% over 2048 paired seeds, on a seed not used for training or selection. Only the exported policy and policy/README.md change.

Contribution checkpoint

• Equivalent of scripts/check.sh: engine build, all tests, simulation and policy smoke (see note)
• scripts/check.sh --with-web: not run, because Emscripten isn't installed on the training host
• scripts/check.sh --with-training: not applicable (no trainer or network changes in this PR)

Note: training ran on Windows, where this tree doesn't build with MSVC, so scripts/check.sh couldn't run as is. On this branch I ran upstream's JS and about-data tests directly. The engine gates ran on the same engine built with MSVC from my fork: native tests, simulate 1000 42 4, web-smoke, and static-policy-smoke on the new JSON. CI here will run the real gate.

Check output (last lines)

pass tests/web_card_back_assets_tests.js
pass tests/web_card_fx_tests.js
pass tests/web_history_tests.js
pass tests/web_static_engine_tests.js
pass tests/web_targeting_tests.js
pass tests/web_voxel_models_tests.js
pass tests/web_voxel_stage_tests.js
pass tests/about_data_tests.py
all tests passed
games=1000 law_wins=559 outlaw_wins=441 decisions=184121
static-policy-smoke policy/static-policy.json -> {"ai":{"iteration":2861,"trained":true},"ok":true,"phase":"play"}

Trained policy submission

• vs current champion: 2293 / 4096 paired games (55.98%, 95% Wilson [54.5%, 57.5%]), seed 1300001, 2048 pairs. Law 1065/2048, Outlaw 1228/2048, 0 truncated.

• vs heuristic baseline: 4082 / 4096 (99.66%). The current champion scores ~99.4%.

• Exact command run:

  ./build-train/bang_duel evaluate runs/mmd_v2_a030_s2/champion/policy.pt policy/static-policy.json 2048 1300001 6
  ./build-train/bang_duel evaluate runs/mmd_v2_a030_s2/champion/policy.pt - 2048 1300001 6

  The opponent in the first command is the current policy/static-policy.json, because I don't have the iteration-2995 .pt. My fork's evaluate accepts a .json opponent and plays it greedily through StaticPolicyLogits. I checked that this matches LibTorch: a checkpoint scored against its own export over 5000 positions differs by at most 3e-5 in logits, and only on exact ties in argmax. Re-running against your .pt with this repo's evaluate should reproduce the result, and I'm happy to share the candidate .pt.

• Training config: model v2, about 2.6 hours, 256 batch games, 3000 iterations (--max-chunks 300), host Windows 11 with the PPO update on an RTX 4060 Ti (rollouts and evaluation on CPU, 4 threads). Otherwise the defaults: lr 3e-4, target KL 0.015, 4 epochs, 8 minibatches, entropy 0.01, same league mix. Seed 520001.

• What changed and why you think it helped: the one change is an MMD term in the PPO loss, 0.3 · KL(π ‖ magnet). The magnet starts as uniform play over legal actions and moves to a frozen copy of the learner every 50 updates. Self-play PPO plateaus around 50% against its own recent snapshots, and MMD is designed to stop that kind of cycling in two-player zero-sum imperfect-information games. A controlled comparison supports it, all with the identical recipe on seed 900001, vs the current champion:

  Run	vs current champion
  PPO replica (no MMD)	48.4%
  MMD α=0.03	52.7%
  MMD α=0.1	55.4%
  MMD α=0.3	55.5%
  MMD α=0.3, second seed (this policy)	57.7%
  MMD α=0.5	56.6%
  MMD α=1.0	54.8% (too random; entropy 0.70)

  The two α=0.3 seeds tie head to head (2048/4096), so the gain replicates. Seed 900001 was used to choose α and the checkpoint, which is why the headline numbers above use a fresh seed. The trainer change is in my fork (--mmd-alpha, --magnet-interval in iakraptis/bang); I can open it as a separate PR if you want it upstream.

• Played it and it behaves sensibly. I played the exported JSON through the native web server (bang_duel web --static-policy, same C++ scoring as the WASM build), not through the WASM bundle itself.

• Updated the iteration/source noted in policy/README.md

Tests

None added: this PR changes only the policy artifact. Separately, while doing this I found that StaticPolicyLogits hard-codes the actor block width as 64 (v2's action_hidden), so exported v3 policies score moves incorrectly; v2 is unaffected. The fix is in my fork and I can send it as its own PR.