Skip to content

Version MNG canonical output and add compatibility goldens - #14

Merged
Satsrag merged 2 commits into
Satsrag:mainfrom
satsragclaw:feat/mng-canonical-contracts
Jul 27, 2026
Merged

Version MNG canonical output and add compatibility goldens#14
Satsrag merged 2 commits into
Satsrag:mainfrom
satsragclaw:feat/mng-canonical-contracts

Conversation

@satsragclaw

@satsragclaw satsragclaw commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • freeze the current MNG canonical selection policy as mng-canonical/1
  • expose the loaded policy through MongolianShaper.canonical_version
  • embed the version in MNG.normalize.json and reject mismatched table versions
  • add reproducible compatibility fixtures:
    • 15 ordered shaping-trace vectors covering every MNG shaping-rule transition
    • 1,993 byte-for-byte canonical vectors covering every current corpus shape group
  • add scripts/gen_compat_goldens.py with write and --check modes
  • fail on unknown corpus aliases, freeze vector cardinality, and require each trace vector to trigger its declared witness rule
  • include test data, goldens, and generator scripts in source distributions

Why

Applications may persist normalized search or index keys. A named canonical policy lets them detect a future output-policy change and rebuild those keys deliberately.

The branch is rebased directly onto merged main@3effbc6618b5c521864d78e76bda70a8bc70fd7f. Regenerating after the public PascalCase control-name change updated 784 canonical shape labels and 3 trace shape labels from mvs/nirugu/zwj to Mvs/Nirugu/Zwj; all normalized code points, positions, rule transitions, conditions, and per-token written units remained unchanged.

Verification

  • python3 -m compileall -q mongol_norm scripts tests
  • python3 scripts/gen_normalize_table.py MNG twice, byte-identical output
  • python3 scripts/gen_compat_goldens.py --check and regeneration twice, byte-identical output
  • python3 -m unittest discover -s tests -p 'test_*.py'
    • 186 tests passed on Python 3.11
    • 186 tests passed on Python 3.6.8
    • 177/177 core-hud direct cases
    • 3507/3507 eac-hud direct cases, with 5 documented UTN xfails skipped
    • 225/225 CORE-HUD round trips
    • 3513/3513 EAC-HUD round trips
    • 2237/2237 prefix-stability pairs
    • 1993/1993 shape groups converge
  • uv build succeeded for wheel and sdist
  • installed-wheel smoke passed and exposed mng-canonical/1
  • sdist contains the compatibility generator and both committed golden fixtures
  • two independent file-only reviews passed with no security or logic blockers

Closes #10

@satsragclaw
satsragclaw force-pushed the feat/mng-canonical-contracts branch from a754f02 to f8d4f8d Compare July 26, 2026 13:15
@Satsrag
Satsrag merged commit cd10915 into Satsrag:main Jul 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Version the canonical normalization format to ensure index stability

2 participants