research: add SYM-ARCH-002A2 hierarchical statistics and power planning - #58
Conversation
Static statistics audit — nuisance topology hardeningA post-implementation audit found a dependence-structure ambiguity worth fixing before A2 is allowed to inform CONFIRM sizing. The original hierarchical code correctly treated environments as the independent scientific unit, but it modeled representation/learner/stream realizations as nuisance runs nested inside each environment. If the same nuisance realization is reused across multiple environments, those effects are crossed instead; independently resampling them inside each environment can distort uncertainty. This PR now adds
A dedicated crossed prospective-power simulator remains future work and must be separately validated before a crossed CONFIRM design can use it. The restriction and confirmatory freeze requirements are documented in This is statistical-infrastructure hardening only; no architecture result or sample-size recommendation has been observed or changed. |
Integration-only parent sync; no A2 scientific changes.
Summary
Adds the next focused SYM-ARCH-002A2 statistical-infrastructure tranche from issue #55.
This PR is deliberately stacked on #57 (
research/sym-arch-002a-core-v1) and does not modify #57's frozen conceptual scope. It should remain draft/stacked until #57 lands, then be rebased/retargeted tomainwithout changing the statistical contract.No architecture winner/loser claim and no actual CONFIRM sample-size recommendation are produced by this PR.
Hierarchical inference
Adds
experiment_statisticswith:NestedEnvironmentResultas the independent-world container;PairedRunResultfor candidate/control outcomes under the same nuisance realization;The nested interval is explicitly named percentile rather than silently reusing the existing one-level BCa label.
Prospective power
Adds a DEV-only Monte Carlo power planner that sizes future independent-environment counts against the same SESOI practical-effect gate introduced by #57.
Important safeguards:
DEVmanifest may drive planning;planning_effectis separately frozen and must lie beyond the SESOI in the requested direction;planning_effect, reducing winner's-curse/tuning bias from the observed DEV mean;residual_scale >= 1.0supports conservative variance-inflation sensitivity analysis;minimum_environmentsis selected only when the lower power bound clears target and remains clear for every larger tested count.If the tested count grid does not satisfy that sustained rule, the result is
None; the target is not weakened after inspection.Provenance
The power plan records:
Changing data, planning effect, SESOI, variance inflation, counts, simulation settings, or seed changes the planning-input digest.
Tests
Focused tests cover:
CI
Adds
SYM-ARCH-002A2 Hierarchical Power, which supports both the current stacked base and a latermainretarget. It runs:experiment_statisticstests;cargo check -p symthaea-psych-bench --lib.Scientific boundary
This PR provides measurement/planning infrastructure only. It does not choose a planning effect, does not run a DEV campaign, does not open CONFIRM, and does not support any Symthaea capability claim.
The planning effect must be justified and frozen outside the code path. Multiple-comparison correction, procedural-world mutation tests, resource Pareto analysis, diagnostic state interventions, and stronger baselines remain separate later tranches.
Depends on #57. Tracks #55.