Deep repo review, SOTA research, and metaharness Darwin/flywheel integration

[ruvnet]

Summary

Concurrent agent-swarm workflow (hierarchical topology; 14 specialized agents across five waves): deep technical review of ruv-FANN, 2026 SOTA landscape research, metaharness integration design, a working OpenRouter flywheel + Darwin optimization harness with two completed live evolution campaigns, a hardening wave (CI gating, security audit + remediation, benchmark baseline, dependency/claims modernization) — and a measured two-stage forward-pass optimization culminating in a SoA GEMV fast path: MNIST-sized inference 127.3µs → 19.7µs (6.5x).

⚠️ Security notice (action required by maintainers)

The tracked .claude.json (~1.7 MB) contained live third-party credentials (Supabase access token + password, ElevenLabs API key, Google Gemini API key) inside captured session logs. This PR removes it from the index and gitignores it, but the values remain in git history on main. Rotate all four credentials, then purge history (e.g. git filter-repo). Details in docs/research/06-security-audit.md §5 (report is fully redacted).

Performance (all measured, criterion --sample-size 10 --measurement-time 2, reproduced across runs)

Forward bench	Session baseline	After stage 1 (alloc/hoist)	After stage 2 (SoA GEMV)	Total
XOR [2,3,1]	106.7 ns	88.6 ns	95.1 ns	1.12x (see note)
Small [10,20,10]	867.1 ns	566.5 ns	374 ns	2.3x
Medium [100,50,25,10]	7.87 µs	5.73 µs	1.59 µs	4.9x
MNIST-sized [784,128,64,10]	127.3 µs	110.6 µs	19.7 µs	6.5x

• Stage 1: scratch-buffer reuse, single-bounds-check accumulation, per-layer activation hoist (no layout change).
• Stage 2 (src/soa.rs, per the measured plan in docs/research/09): derived row-major SoA weight matrix per dense layer, 4-accumulator GEMV kernel with fused activation, lazy dirty-flag rebuild — AoS stays the source of truth, so training and introspection are unchanged; sparse/cascade topologies keep the original path. No unsafe, no new dependencies. Equivalence test: max relative delta 3.39e-7 vs the per-neuron path. Training benches unchanged-to-slightly-faster.
• Note: XOR-sized nets pay ~6ns plan-guard overhead (−7% from stage-1 best, still faster than baseline) — documented tradeoff, tunable with a size threshold if desired.
• Kernel choice driven by measured micro-benchmarks (docs/research/09): on 128×784 GEMV, in-house SoA-4acc hits 0.161 ns/MAC native and beats the gemm crate 2.3x on wasm32+simd128 (0.158 vs 0.361); Candle-as-backend rejected for tiny shapes (dispatch/alloc overhead, heavy deps). wasm32 +simd128 rustflags added; cargo check --target wasm32-unknown-unknown passes.

Research (docs/research/)

01 repo deep review · 02 SOTA landscape 2026 (~40 sources) · 03 metaharness integration design · 04 Darwin round-1 (honest null; Goodhart failure caught & corrected) · 05 round-2 island-model evolution (12 candidates, 0 promotions — genome locally optimal; harmful mutation rejected on merit) · 06 security audit (redacted) · 07 benchmark baseline (31 benches, hotspot analysis) · 09 Candle/WASM kernel analysis (measured GEMV micro-benchmarks driving the stage-2 design).

Optimization harness (scripts/optimization/, config/optimization/)

flywheel.js tiered OpenRouter eval runner + darwin-evolve.js elitist genome evolution (safety-gated, island-capable) + 11-task discriminating suite. OPENROUTER_API_KEY from env only; eval logs gitignored. Live campaign spend ~$0.09, zero API failures.

CI & security

• pull_request triggers added to ci.yml/comprehensive-testing.yml (PRs were previously ungated) with blocking root-crate pr-gate; new weekly security.yml (cargo-deny advisories + npm audit).
• Audit: 17 Rust advisories in ruv-swarm/Cargo.lock (incl. rmcp DNS rebinding HIGH 8.8), fixable ws/tar-fs npm prod vulns, 10 undocumented unsafe AVX2 sites (now accurately described in README); root-workspace advisories remediated by this PR's lockfile update.

Modernization

rand 0.9 / thiserror 2 / getrandom 0.3 (+wasm_js cfg) / ~90 transitive bumps (wgpu, bincode 2, criterion deliberately held back). README falsifiable claims corrected (unsafe inventory, SWE-Bench provenance footnote, model count 27+→17). Semver note: randomize_weights trait-bound path changed; Network gained a private serde-skipped cache field (struct-literal construction from outside the crate no longer possible; builder/new unaffected).

Test plan

• cargo test --lib --tests: 177/177 passing at final state (173 baseline + 4 new SoA tests), verified independently by agents and coordinator
• Numerical equivalence test old-vs-new forward path (≤1e-4 asserted, 3.39e-7 observed)
• Before/after benches reproduced across two runs at each stage; one regressing attempt measured and reverted
• cargo check clean incl. --features gpu and --target wasm32-unknown-unknown
• Workflow YAML validated; pr-gate should become a required status check after merge
• Pre-existing failures (untouched, documented): two broken examples; ruv-swarm + neuro-divergent workspace manifests unbuildable