Skip to content

The trained walker actually walks - #3

Merged
abgnydn merged 3 commits into
mainfrom
fix/trained-walker-walks
Jul 31, 2026
Merged

The trained walker actually walks#3
abgnydn merged 3 commits into
mainfrom
fix/trained-walker-walks

Conversation

@abgnydn

@abgnydn abgnydn commented Jul 31, 2026

Copy link
Copy Markdown
Owner

The published Vaxenburg walking policy now walks the flybody under real physics with the kinematic assist off. Four port defects, each a silent disagreement with upstream flybody, each verified against upstream source.

Measured

tools/walkbench.mjs, kinematic assist explicitly off, 3 reps:

before after
capsized every rep never, 3/3
uprightness at end −0.78 (on its back) +0.997
speed 0.15 cm/sim s (tumbling) 2.004–2.021 cm/sim s
|action|max while upright 17.8–83.2 ~5.9 (native band ~6)

Commanded speed was 2.0 cm/s — tracking within 1%. Control, policy disabled, assist off: 0.032 cm/sim s. The fly stands still, so the locomotion comes from the policy and nothing else.

The four defects

  1. world_zaxis read the wrong axis of the rotation matrix — the third column where flybody reads the third row (fruitfly.py: MJCFFeature('xmat', root_body)[6:]). Transposes, so pitch and roll were sign-inverted, on what is by input-weight L2 norm the 2nd-highest-gain row of the input layer. The righting response added to a tilt. Independently confirmed against the shipped normalization stats: mean(accelerometer) ÷ mean(world_zaxis) = 981, exactly the gravity in fruitfly.xml.

  2. The spawn height ignored the floor. floor.xml puts the plane at -0.15; _SPAWN_POS is calibrated against a plane at 0. The lowest claw started 0.15 cm up instead of 0.00048, so every episode opened with ~17.5 ms of free fall, claws off the ground and touch/force observations reading zero, exactly while the policy was establishing its gait.

  3. 455 of 741 observation dims never read the body. Both branches differenced the reference against itself, so a tracking controller was told "zero error, perfectly aligned" on every tick, right through the capsize. Now Rᵀ_fly·(refPos − flyPos) and conj(q_fly) ⊗ refQuat against the live pose, indexed from f=0, over an absolute world trajectory the fly can fall behind.

  4. flybody's actuator filter dynamics were missing. Upstream applies dyntype='filter' programmatically in fruitfly.py:_build, so it never reaches the shipped MJCF and a port reading the MJCF cannot see it. model.na was 0 — every control tick drove an actuator 100% to target instead of 18%, a plant ~5.5× stiffer than the one the policy was trained on, and the 59-dim actuator_activation observable was all zeros. Patched into the MJCF text in memory (the runtime reads it from the baked bundle, not from public/), asserted via model.na === 78 so a silent no-op throws.

Also

The kinematic assist is cleared under the policy again, restoring what a9e24f1 reverted. That revert was correct then — removing the assist from a broken walker made the feature visibly fail. It walks now, so the policy path no longer needs it. The CPG path still does, and every "assist off = pirouettes in place" measurement in LIMITATIONS §8 remains true for it and is now labelled as such.

The RL test gates on real locomotion instead of finiteness. Threshold 0.5 cm/sim s sits 4× below measured and 15× above the policy-disabled floor; the pre-fix value of −1.174 cm fails outright.

Not changed

  • The connectome still does not walk the body. Brain→spine still yields a walking magnitude and turn bias scaling a hand-written tripod CPG; ~20.3M edges still reach the body as about one scalar per tick. What works is a published RL policy, not the fly's brain.
  • The forward pass has still never been compared against the released SavedModel. verify_walking_policy.py imports no TensorFlow. The walker working is strong circumstantial evidence the architecture guess is right; it is not that comparison.

Verification

tsc clean · unit 5/5 · full e2e 34/34 on the baseline containing fixes 2 and 4. Two later full runs were contaminated by unrelated load on this machine (11 vitest workers from another session; 15-min load average 10.78; tests taking 6–8 min against a normal 40 s). Those runs failed 3 and 4 tests respectively with zero overlap, and all 7 pass individually on an idle machine — the signature of load, not regression. The RL test itself passed 6 separate single-test runs at 2.016–2.020 cm/sim s.

🤖 Generated with Claude Code

abgnydn and others added 3 commits July 31, 2026 16:54
Four places where this port disagreed with upstream flybody. Each was
silent, each had zero error at spawn and grew from there, and together they
meant the published Vaxenburg policy could not walk the body — the demo's
forward motion was the kinematic assist gliding a falling fly.

1. world_zaxis read xmat's third COLUMN where flybody reads the third ROW
   (fruitfly.py: MJCFFeature('xmat', root_body)[6:]). Those are transposes,
   so pitch and roll were sign-inverted on what is, by input-weight L2 norm,
   the 2nd-highest-gain row of the 741x512 input layer. The fly's righting
   response therefore added to a tilt instead of opposing it. Confirmed
   against the shipped normalization stats: mean(accelerometer) /
   mean(world_zaxis) = 981, exactly fruitfly.xml's gravity.

2. The spawn height ignored the floor. floor.xml puts the plane at
   pos="0 0 -.15"; flybody's _SPAWN_POS is calibrated against dm_control's
   floors.Floor() at z=0. The lowest claw started 0.15 cm up instead of
   0.00048, so every episode opened with ~17.5 ms of free fall, all six
   claws off the ground and the touch/force observations reading zero,
   while the policy was establishing its gait. Spawn is now floor-relative,
   with the plane read out of the compiled model.

3. 455 of the 741 observation dims never read the body. Both branches of
   the ref block differenced the reference against ITSELF, so a tracking
   controller was told "zero error, perfectly aligned" on every tick,
   straight through the capsize. ref_displacement is now
   R^T_fly·(refPos − flyPos) and ref_root_quat is conj(q_fly) ⊗ refQuat,
   both against the live pose, indexed from f=0 so entry 0 is the current
   error, over an absolute world-space trajectory the fly can fall behind.

4. flybody's actuator filter dynamics were missing entirely. Upstream sets
   dyntype='filter' programmatically in fruitfly.py:_build, so it never
   appears in the shipped MJCF and a port that reads the MJCF cannot see
   it. model.na was 0: every control tick drove an actuator 100% of the way
   to its target instead of 1-exp(-0.002/0.01) = 18%, a plant ~5.5x stiffer
   than the one the policy was trained against, and the 59-dim
   actuator_activation observable was all zeros. Patched into the MJCF text
   in memory at load (the runtime reads fruitfly.xml from the baked bundle,
   not from public/), asserted via model.na === 78 so a no-op patch throws.
   data.act is now indexed through actuator_actadr, which is only correct
   once na > 0.

Measured with tools/walkbench.mjs, kinematic assist explicitly off, 3 reps:
2.004-2.021 cm per simulated second against a 2.0 cm/s command, uprightness
+0.997, never capsizing, actions ~5.9 while upright (flybody's native band
is ~6). Before: 0.15 cm/sim s, on its back in every rep. Control with the
policy disabled: 0.032 cm/sim s — it stands still, so the locomotion comes
from the policy and nothing else.

Target speed goes 1.0 -> 2.0 cm/s to match upstream's inference default.
The old comment attributed the 1.0 tuning to observation-normalization
drift; the real cause was defect 3.

The kinematic assist is also cleared under the policy now, restoring what
a9e24f1 reverted. That revert was right at the time — removing the assist
from a broken walker made the feature visibly fail. It walks now, so the
policy path no longer needs it. The CPG path still does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The RL walker test asserted only that the policy ticked and its actions
were finite, with a comment explaining that forward progress was
deliberately not asserted because the walker did not walk. Both are now
stale.

It closes its window on the sim clock and asserts net displacement per
simulated second plus end-of-window uprightness, matching the neighbouring
walk-gate test. Threshold 0.5 sits 4x below the measured 2.016-2.020 and
~15x above the policy-disabled floor of 0.03, and the pre-fix value of
-1.174 cm fails outright — so a regression of any of the four fixes is
caught here, and a regression of the actuator filter fails even earlier at
the model.na assertion in create().

|action|max is deliberately not gated: __rlActionStats starts accumulating
when the toggle is clicked, before the test's reset, so it captures the
handover transient and has been logged at 6.3, 65.8 and 166.2 across runs
that were otherwise identical. Locomotion was unaffected in all three.

tools/walkbench.mjs is the harness the LIMITATIONS numbers come from. It
disables the kinematic assist explicitly and samples time-resolved, because
every quantity worth knowing is "while the fly is still upright" — a
running max over the whole window is dominated by the post-capsize blow-up
and made the policy look far worse than it was. Committing it so the
published numbers are reproducible rather than asserted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two days ago these files were made honest on the evidence that the trained
walker did not walk. That specific fact has changed, so the claim has to
change with it.

LIMITATIONS.md item 1 was "Trained RL walker does not walk". It now records
what is measured — ~2.0 cm per simulated second against a 2.0 cm/s command,
upright +0.997, no capsize across 3 reps with the kinematic assist off —
and, more usefully, why it used to fail: four port defects, each a silent
disagreement with upstream flybody.

Section 8's inventory is qualified rather than rewritten. The assist row,
and every "assist off = the fly pirouettes in place" measurement, is now
explicitly labelled as the CPG path, where it remains true. The row about
the assist being live under the policy is stale and says so.

Deliberately unchanged, because none of it is affected:
- The connectome still does not walk the body. Brain to spine still yields
  a walking magnitude and a turn bias that scale a hand-written tripod CPG;
  ~20.3M edges still reach the body as about one scalar per tick. What
  works now is a published RL policy, not the fly's own brain.
- The forward pass has still never been compared against the released
  SavedModel. tools/verify_walking_policy.py imports no TensorFlow. The
  walker working is strong circumstantial evidence the architecture guess
  is right, but it is not that comparison, and the caveat stays.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@abgnydn
abgnydn merged commit 14c1ecf into main Jul 31, 2026
1 check passed
@abgnydn
abgnydn deleted the fix/trained-walker-walks branch July 31, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant