[nichinichisou0609]

Problem

The Python whole-chart cache invalidates unaffected work when a catalog or global dependency changes and loses completed subcomputations after a later failure. It also retains the sampled chart-stats/2 contract while the current native model computes nominal chart-stats/3 expectations.

Changes

• Replace Python chart/P0/profile caches with the native content-addressed program cache from feat(chartstats): cache compiled measurement programs empty-sekai/ournotes-deck#44. Pin the ournotes-deck v0.0.5 source commit 67835811e06f3d528adb90ae64b012a0eb5f8966; the cache has separate expectation, deterministic-run and lottery-curve units.
• Expose cached chart statistics through the PyO3 bridge, validate the complete original input before cache use, share the cache across workers and retain deterministic output order. Report actual native requests, hits, computations, writes, invalid records and written bytes.
• Publish nnnotes.music-data/2 with the exact chart-stats/3 contract. Separate nominal expectations and numerical enclosures from replay seeds; remove sampled aptitude caps and the old cache migration paths.
• Update JSON Schema, exporter validation, publication gates and page smoke tests. Check full shape/range dimensions, interval identities, deterministic counters and independent nominal check reconstruction; unsupported rank transformations remain explicitly unavailable.
• Start a new regional Actions cache namespace and preserve each completed native record after later failures. Pin the compatible page consumer (ournotes-player commit 44d3175, ci(music-data): keep chart statistics between builds #18); configured overrides must pass the new-format smoke test before the expensive build.
• Update CLI, format, replay and rollout documentation.

Validation

• Full default Python suite with the actual native extension installed: 1,377 passed, 5 skipped. Skips are one optional freetype dependency and four existing golden/library-version checks.
• Focused musicdata/statscache/replaydata suites: 178 passed, no skips; included in the full suite above.
• Catalog/JP/story/recognition workflow suites: 115 passed, 1 optional onnxruntime skip.
• Publication, JP, refresh, compression and prebuilt identity suites: 217 passed, 2 optional external-sample skips, with the real compatible page and native chart-stats/3 fixture.
• The real 352-chart output passes the deck, aptitude and gzip gates.
• Full source/test pyflakes, CLI help, Rust fmt, Rust clippy for all targets with -D warnings, and git diff checks pass.

Real PyO3 cache checks

• Three real charts covering Just, Combo and Luck produce byte-identical uncached, cold and warm JSON.
• Warm reuse: 1,023 hits, zero computations and zero writes, including a change in worker count.
• Replay seed prefix growth from 2 to 16 and provenance-only changes perform no new expectation computations.
• Seven classes of malformed original master/chart inputs are rejected even with a populated cache, without changing cache bytes.
• A later-chart failure preserves 442 completed records; the repaired retry reuses 616 operations and computes only 582 units, versus 1,024 for a cold run.
• Corrupting one expectation entry causes exactly one invalid record, one computation and one write; final output remains byte-identical.
• Native cache IO errors become MusicDataError instead of silently disabling the cache.

The native PR additionally verifies an entire 348-to-352-chart update: 95,940 cache hits and 6,644 new computations; the next warm run has 100,676 hits and zero new computations. Both outputs match uncached full recomputation byte for byte.

Coordinated rollout

The page consumers are empty-sekai/ournotes-player#18 and StarMoe-org/moenotes#38; both read the new format and keep sampled files readable. The songs page files are published from the pinned player commit. The publisher selects engines by the package version in Cargo.lock and checks their actual commit, source and file identities against the v0.0.5 release.

This PR does not publish production data by itself; data is published by the music-data workflow.

🤖 Generated with Claude Code