research: add SYM-ARCH-002B1 strong-simple analytic baselines - #62
Conversation
Tristan-Stoltz-ERC
left a comment
There was a problem hiding this comment.
Static scientific-fairness audit hardening completed on the current B1 branch.
Two distinct gaps were closed:
-
Contrast interpretation: sharing one RLS implementation does not make one-hot↔random/HDC a readout-shape-matched contrast when effective feature dimensions differ.
experiment_baseline_fairnessnow audits schema, RLS protocol, effective readout/covariance shape, and paired randomized seed index, returning onlyrepresentation_levelorreference_only. Random↔HDC is the intended clean representation contrast in the matched family; dimension-mismatched one-hot comparisons are explicitly capacity/reference comparisons. -
Matched exposure: static specs do not prove that a runner actually showed each baseline the same data/order/update opportunities.
MatchedBaselinePanelnow feeds training observations to all three agents atomically, records separate order-sensitive training/evaluation stream digests, validates equal update counts, records member spec/resource snapshots, and exposes a canonical panel-snapshot digest for evidence binding. Failed training/evaluation items leave the panel ledger unchanged.
The B1 workflow now runs rustfmt plus focused tests for baseline mechanics, fairness auditing, and the matched panel before the psych-bench library check.
No performance result or HDC advantage is claimed by these changes.
Summary
Adds SYM-ARCH-002B1 strong-simple analytic baselines for the architecture-discrimination program in #55.
This is a draft sibling of #60/#61, stacked directly on #59 / A3. It does not consume CONFIRM data and carries no architecture-performance claim.
The goal is to attack a simple alternative explanation before adding GRU, trainable SSM, Mamba, liquid dynamics, or Hebbian plasticity:
Baseline ladder
B1 adds three deterministic conditions:
one_hot_rls— normalized categorical one-hot representation + full-covariance online RLS.fixed_random_tanh_rls— deterministic fixed random projection + tanh/L2 normalization + the same RLS algorithm/configuration.vanilla_hdc_rls— deterministic role/value HDC binding/bundling + normalization + the exact same RLS readout shape as the random condition at matched encoded dimension.MatchedBaselineFamilySpecemits all three from one schema/RLS contract.Important terminology hardening: sharing the same RLS algorithm/configuration does not automatically mean two conditions are readout-shape matched. One-hot usually has a smaller effective feature dimension, so its trainable RLS state and O(d^2) covariance can differ from random/HDC.
Executable contrast-fairness audit
experiment_baseline_fairnessrecords exact spec digests, learner-visible schema equality, RLS contract equality, effective feature/readout/covariance shape, and paired representation-seed index when both encoders are randomized.It returns only:
representation_levelwhen a representation-only interpretation is admissible;reference_onlywhen capacity/protocol differs enough that such attribution is not clean.Random↔HDC is the intended representation-level B1 contrast. Dimension-mismatched one-hot↔random/HDC is explicitly a lower-complexity reference/capacity comparison. Therefore
random > one_hotis not, by itself, proof that nonlinear random expansion caused the improvement if RLS capacity also increased.The audit does not require equal fixed-encoder storage; resource efficiency is reported separately.
Matched data and update opportunities
experiment_baseline_panel::MatchedBaselinePanelis the preferred claim-bearing execution path.It owns all three baseline agents and:
If any model rejects a training or evaluation item, no exposure ledger is partially committed. Claim-bearing B1 results should bind the valid panel snapshot digest; a hand-written loop over independent agents is not sufficient evidence for a matched-stream claim.
RLS / resource contract
OnlineRlsBinaryuses {-1,+1} targets and standard recursive least squares with frozen ridge/forgetting/bias policy, no replay buffer, no learned encoder, and no temporal state.Replay-free is not memory-free: full f64 inverse covariance is O(d^2) state.
BaselineResourceFootprintreports feature dimension, encoder bytes, readout weights, covariance bytes, total persistent state, trainable parameters, replay examples (0), and temporal-state bytes (0).A hard 512 MiB covariance safety ceiling rejects oversized full-RLS requests before allocation. Full RLS therefore cannot be silently applied at Symthaea's ordinary 16K HDC dimension.
Frozen schema / provenance
The categorical schema is strict: normalized/sorted unique names, explicit sorted domains, exact learner-visible feature set, fail-closed out-of-domain values, and normalized one-hot representation.
Each emitted baseline spec has a domain-separated BLAKE3 digest over its versioned configuration. That digest does not pretend to bind a git commit; exact source identity is separately frozen in the experiment manifest
code_revisionand later claim/evidence binding.Scientific interpretation boundary
An HDC win does not isolate HDC binding algebra. Exact continuous-vs-bipolar/BinaryHV binding attribution remains a later matched factorization/algebra tranche.
The implementation is RanDumb/F-OAL-inspired, not an exact reproduction of either external method.
Tests / CI
The dedicated workflow now gates:
Wording ceiling
Merging this PR supports only:
It does not support a performance result, an HDC-vs-random conclusion, a representation-only interpretation for a
reference_onlycontrast, a matched-stream claim without a valid panel snapshot, a named-method reproduction claim, or any conclusion about liquid/Hebbian mechanisms.Depends on #59. Sibling of #60 and #61. Tracks #55.