Full disclosure: make every claim true, inventory every shortcut - #2
Merged
Conversation
The repo claimed the connectome drives the body. It does not, and the measurements are not close. What is actually there: ~20.3M connectome edges reach the body as about one scalar magnitude plus a turn bias per tick, and those two numbers scale a hand-written tripod CPG whose leg phase is sin(sim_time · 10 Hz). The brain → spine path is genuinely a real LIF cascade across two real connectomes with nothing learned or scripted in it; below the spine it is an approximation. README, index.html, CITATION.cff and .zenodo.json all said otherwise. Measured over 16 controlled browser runs: the kinematic assist IS the locomotion. ~0.80-0.88 cm per simulated second with it on, and that number is the same whether the CPG is driving, the trained policy is driving, or nothing is. 0.029-0.163 cm/sim s with it off, where the fly pirouettes in place (3.43 cm of path, 0.17 cm of net travel, -245° of yaw). One run translated at 0.878 cm/sim s, straightness 0.93, while upside down. LIMITATIONS.md gains a full inventory: thirteen shortcuts, nine of which the "Honest mode" button does not touch. Largest undisclosed one is a pitch/roll damper (qvel[3] *= 0.85 per substep) that sits outside the assist guard, so the body cannot tip — and has never once been run in the off state. Also corrected: the walking policy is NOT verified against the published SavedModel. tools/verify_walking_policy.py imports no TensorFlow; it re-runs the same assumed architecture over the same extracted weights, so it validates the port's arithmetic and not the architecture guess. Two behaviour fixes, both about making the demo mean what it shows: - The boot auto-stim left the drive saturated forever (decayDrive() is defined and never called), so a page nobody clicked travelled 1.831 cm and clicking DNa01 travelled 1.838 cm — a 0.4% difference. The boot preset now returns the drive to rest when its window ends. A user-clicked stim still persists. - The "does it walk" gate asserted speed > 0.1 read from bodySpeed = |qvel[0..1]| — the exact slots the assist writes immediately before mj_step, so it read back the command just written. It now measures net displacement per simulated second and requires the fly to still be upright. Removing the boot residual made the RL walker test fail honestly (dx = -1.174 cm), because that test only ever passed on the stale saturated command the assist was gliding on. It now gates what its name claims — the policy ticks and its actions reach the actuators — and no longer asserts forward progress the walker does not produce. e2e 34/34 against the real connectome binaries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #1. The question was whether the product could run with zero cheats. The investigation says no — and the reason is more fundamental than the kinematic assist, so this PR takes the other path: keep the product working exactly as it does today, and make every claim it makes true.
What the measurements showed
16 controlled browser runs, fresh page load per configuration, real connectome binaries.
The kinematic assist is the locomotion. ~0.80–0.88 cm per simulated second with it on — and that number is essentially the same whether the hand-coded CPG is driving, the trained RL policy is driving, or nothing is. 0.029–0.163 cm/sim s with it off. One run translated at 0.878 cm/sim s with straightness 0.93 while upside down: displacement is fully decoupled from body state.
Assist off is not a slow walk — the direction is wrong. The CPG accumulates 3.43 cm of path for 0.17 cm of net displacement, straightness 0.048, total yaw −245°. The fly pirouettes in place. Under the trained policy it capsizes within ~1.5 s.
The information bottleneck, which is the real answer to "does the connectome drive the body":
~20.3M connectome edges reach the body as about one scalar magnitude plus a turn bias per tick. The connectome scales the gait; it does not generate the rhythm. In the "Track target" demo its contribution to locomotion is zero — MANC is skipped and both axes come from retinal geometry.
Claims corrected
tools/verify_walking_policy.pyimports no TensorFlow — it re-runs the same assumed architecture over the same extracted weights the extractor wrote, validating the port's arithmetic but not the architecture guess (whose own comment reads"layernorm bias (?) — actually unsure").LIMITATIONS.mdclaimed the assist is off in RL mode. It isn't:fwdCmd/turnCmdare written only bydriveLegs, which the policy path skips, and nothing zeroes them.What is not walked back: the connectomes are real, both LIF nets genuinely run on the GPU every frame, and the brain→spine cascade is real biology with nothing learned or scripted in it.
New:
LIMITATIONS.md§8Thirteen shortcuts, nine behind no toggle at all, each with what it substitutes for, whether Honest mode disables it, and a file:line. Biggest undisclosed one is a pitch/roll damper (
qvel[3] *= 0.85per substep, outside the assist guard) meaning the body cannot tip — and it has never been run in the off state, so the undamped baseline is unmeasured.Two behaviour fixes
fwdCmd = 0.99999forever (decayDrive()is defined and never called). A page nobody clicked travelled 1.831 cm; clicking DNa01 travelled 1.838 cm — 0.4%. The headline interaction was a no-op in the forward axis. The boot preset now returns the drive to rest; user-clicked stims still persist.speed > 0.1frombodySpeed = |qvel[0..1]|— the exact slots the assist writes immediately beforemj_step, so it read back the command just written. Now measures net displacement per simulated second (0.882 measured vs 0.3 threshold) and requires the fly to still be upright.Removing the boot residual made the RL walker test fail honestly (
dx = -1.174 cm) — it had only ever passed on the stale saturated command. It now gates what its name claims (the policy ticks, its actions reach the actuators) instead of forward progress the walker does not produce.Verification
tsc --noEmitclean ·npm run test:unit5/5 ·npm run test:e2e34/34 against real connectome binaries.🤖 Generated with Claude Code