Skip to content

research: add SYM-ARCH-002B1 strong-simple analytic baselines - #62

Draft
Tristan-Stoltz-ERC wants to merge 20 commits into
research/sym-arch-002a3-validity-v1from
research/sym-arch-002b1-simple-baselines-v1
Draft

research: add SYM-ARCH-002B1 strong-simple analytic baselines#62
Tristan-Stoltz-ERC wants to merge 20 commits into
research/sym-arch-002a3-validity-v1from
research/sym-arch-002b1-simple-baselines-v1

Conversation

@Tristan-Stoltz-ERC

@Tristan-Stoltz-ERC Tristan-Stoltz-ERC commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds SYM-ARCH-002B1 strong-simple analytic baselines for the architecture-discrimination program in #55.

This is a draft sibling of #60/#61, stacked directly on #59 / A3. It does not consume CONFIRM data and carries no architecture-performance claim.

The goal is to attack a simple alternative explanation before adding GRU, trainable SSM, Mamba, liquid dynamics, or Hebbian plasticity:

Is an apparent architecture advantage already explained by a fixed representation plus a strong online analytic readout?

Baseline ladder

B1 adds three deterministic conditions:

  1. one_hot_rls — normalized categorical one-hot representation + full-covariance online RLS.
  2. fixed_random_tanh_rls — deterministic fixed random projection + tanh/L2 normalization + the same RLS algorithm/configuration.
  3. vanilla_hdc_rls — deterministic role/value HDC binding/bundling + normalization + the exact same RLS readout shape as the random condition at matched encoded dimension.

MatchedBaselineFamilySpec emits all three from one schema/RLS contract.

Important terminology hardening: sharing the same RLS algorithm/configuration does not automatically mean two conditions are readout-shape matched. One-hot usually has a smaller effective feature dimension, so its trainable RLS state and O(d^2) covariance can differ from random/HDC.

Executable contrast-fairness audit

experiment_baseline_fairness records exact spec digests, learner-visible schema equality, RLS contract equality, effective feature/readout/covariance shape, and paired representation-seed index when both encoders are randomized.

It returns only:

  • representation_level when a representation-only interpretation is admissible;
  • reference_only when capacity/protocol differs enough that such attribution is not clean.

Random↔HDC is the intended representation-level B1 contrast. Dimension-mismatched one-hot↔random/HDC is explicitly a lower-complexity reference/capacity comparison. Therefore random > one_hot is not, by itself, proof that nonlinear random expansion caused the improvement if RLS capacity also increased.

The audit does not require equal fixed-encoder storage; resource efficiency is reported separately.

Matched data and update opportunities

experiment_baseline_panel::MatchedBaselinePanel is the preferred claim-bearing execution path.

It owns all three baseline agents and:

  • applies each labeled training item to all three atomically using clone-before-commit semantics;
  • records an order-sensitive training-stream digest;
  • evaluates all three on the same labeled evaluation item without model-state mutation;
  • records a separate order-sensitive evaluation-stream digest including the expected label;
  • validates that every member update count equals the panel training-observation count;
  • captures member spec digests and resource footprints;
  • emits a canonical panel-snapshot digest independent of member serialization order.

If any model rejects a training or evaluation item, no exposure ledger is partially committed. Claim-bearing B1 results should bind the valid panel snapshot digest; a hand-written loop over independent agents is not sufficient evidence for a matched-stream claim.

RLS / resource contract

OnlineRlsBinary uses {-1,+1} targets and standard recursive least squares with frozen ridge/forgetting/bias policy, no replay buffer, no learned encoder, and no temporal state.

Replay-free is not memory-free: full f64 inverse covariance is O(d^2) state. BaselineResourceFootprint reports feature dimension, encoder bytes, readout weights, covariance bytes, total persistent state, trainable parameters, replay examples (0), and temporal-state bytes (0).

A hard 512 MiB covariance safety ceiling rejects oversized full-RLS requests before allocation. Full RLS therefore cannot be silently applied at Symthaea's ordinary 16K HDC dimension.

Frozen schema / provenance

The categorical schema is strict: normalized/sorted unique names, explicit sorted domains, exact learner-visible feature set, fail-closed out-of-domain values, and normalized one-hot representation.

Each emitted baseline spec has a domain-separated BLAKE3 digest over its versioned configuration. That digest does not pretend to bind a git commit; exact source identity is separately frozen in the experiment manifest code_revision and later claim/evidence binding.

Scientific interpretation boundary

  • HDC ≈ random under a passing fairness audit → no evidence beyond generic fixed random features.
  • HDC > random under a passing fairness audit → representation-level evidence for the factorized HDC implementation.
  • random > HDC → HDC is less useful in that regime.
  • one-hot comparisons remain simpler-model/reference contrasts whenever the fairness audit finds a capacity mismatch.

An HDC win does not isolate HDC binding algebra. Exact continuous-vs-bipolar/BinaryHV binding attribution remains a later matched factorization/algebra tranche.

The implementation is RanDumb/F-OAL-inspired, not an exact reproduction of either external method.

Tests / CI

The dedicated workflow now gates:

  • rustfmt for all B1 Rust paths;
  • baseline mechanics/RLS/resource tests;
  • contrast-fairness tests;
  • matched-panel stream determinism, atomicity, update-count, label-binding, and canonical snapshot-digest tests;
  • full psych-bench library check.

Wording ceiling

Merging this PR supports only:

Symthaea psych-bench contains deterministic, resource-audited one-hot, fixed-random, and vanilla-HDC online-RLS baselines plus executable contrast-fairness and matched-exposure provenance suitable for later preregistered comparisons.

It does not support a performance result, an HDC-vs-random conclusion, a representation-only interpretation for a reference_only contrast, a matched-stream claim without a valid panel snapshot, a named-method reproduction claim, or any conclusion about liquid/Hebbian mechanisms.

Depends on #59. Sibling of #60 and #61. Tracks #55.

@Tristan-Stoltz-ERC Tristan-Stoltz-ERC left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static scientific-fairness audit hardening completed on the current B1 branch.

Two distinct gaps were closed:

  1. Contrast interpretation: sharing one RLS implementation does not make one-hot↔random/HDC a readout-shape-matched contrast when effective feature dimensions differ. experiment_baseline_fairness now audits schema, RLS protocol, effective readout/covariance shape, and paired randomized seed index, returning only representation_level or reference_only. Random↔HDC is the intended clean representation contrast in the matched family; dimension-mismatched one-hot comparisons are explicitly capacity/reference comparisons.

  2. Matched exposure: static specs do not prove that a runner actually showed each baseline the same data/order/update opportunities. MatchedBaselinePanel now feeds training observations to all three agents atomically, records separate order-sensitive training/evaluation stream digests, validates equal update counts, records member spec/resource snapshots, and exposes a canonical panel-snapshot digest for evidence binding. Failed training/evaluation items leave the panel ledger unchanged.

The B1 workflow now runs rustfmt plus focused tests for baseline mechanics, fairness auditing, and the matched panel before the psych-bench library check.

No performance result or HDC advantage is claimed by these changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant