From a69202618d1c079293d1e97f0244e52901b0cd05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ahmet=20Bar=C4=B1=C5=9F=20G=C3=BCnayd=C4=B1n?= Date: Mon, 3 Aug 2026 16:35:08 +0700 Subject: [PATCH] fix: correct five false public claims, and measure the attitude damper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four independent hostile reviews of the public surface — connectomics, RL, graphics, community — all returned publish-with-fixes and found five statements that were false. The engineering and the science held up; the risk was concentrated in about six sentences. THE LANDING PAGE NEVER GOT THE CLEANUP. index.html still said a command neuron "named DNa01 in the brain is the same cell as DNa01 in the spinal cord" and that the path is "wire for wire from the real animal", with "nothing learned or scripted". All three are contradicted by this repo's own LIMITATIONS §5: FlyWire is a female FAFB brain and MANC a male VNC, no wire crosses the boundary at all, and what does cross is a rolling firing rate averaged over the DN's L+R copies, scaled by a hand-chosen constant, written as a constant external current. A scalar, not spikes. That page has the widest readership and is the Space's app_file. THE KENYON-CELL CLAIM WAS CIRCULAR. README and LIMITATIONS presented KC sparsity landing in the canonical 5-15% band as the strongest dynamics claim. src/sim.ts says w_syn was "tuned EMPIRICALLY ... to land KC at the canonical 5-15%". The one free synaptic weight was fitted to produce the number then reported as agreement with Shiu et al. 2024, and none of "w_syn", "tuned" or "0.275" appeared in any public document. It is now stated as a calibration target rather than a validation: what it shows is that a w_syn exists putting the network in a plausible regime rather than silent or saturated. THE WORD "alone" WAS FALSE, INCLUDING IN THE DOI RECORD. Five places claimed the trained policy walks "from leg actuation and ground reaction alone" with the kinematic assist off. The pitch/roll damper sits ABOVE the assist's guard in step(), so it runs regardless — and no uprightness figure this project has ever recorded was damper-free, because the damper has been on unconditionally for its whole life. So it is now gated behind Physics.attitudeDamperEnabled and measured. Trained policy, assist off, 3 reps each: damper ON (as shipped) 2.019 cm/sim s uprightness +0.997 damper OFF 0.068 cm/sim s uprightness -0.87 Commanded speed 2.0 cm/s. With the damper off the fly capsizes and stops walking — 0.068 is nearer the 0.032 of a body with no controller at all than the commanded 2.0. Translation is earned: with the assist off nothing writes qvel[0..2], so the forward motion genuinely comes from leg actuation and ground reaction. Attitude is not. The docs now keep those two halves apart instead of collapsing them into one sentence. PROVENANCE AND ATTRIBUTION. space/README.md listed the flybody body model and the trained walking policy under one licence when NOTICE gives them different provenance, omitted MANC's licence, and pointed at NOTICE and LIMITATIONS.md — neither of which deploy:hf uploaded. Both pointers were dead and shipping the mujoco wasm without NOTICE does not satisfy Apache-2.0 §4(d). deploy:hf now ships NOTICE, LICENSE, LICENSE-FLYBODY and LIMITATIONS.md. Also scoped the rhythm negative result to "our deafferented LIF model of MANC" rather than to the connectome — a memoryless LIF network with no adaptation, delays or sensory feedback structurally cannot half-centre oscillate, so the unscoped version is what a referee would reject. Added the deafferented cord (all 6,282 sensory neurons at zero) to the §8 inventory, where it was missing. e2e 34/34, unit 5/5, tsc clean. Co-Authored-By: Claude Opus 5 --- .zenodo.json | 2 +- CITATION.cff | 14 +++-- LIMITATIONS.md | 140 ++++++++++++++++++++++++++++++++++-------------- README.md | 26 +++++---- index.html | 37 ++++++++----- package.json | 2 +- space/README.md | 28 ++++++---- src/physics.ts | 16 ++++-- 8 files changed, 183 insertions(+), 82 deletions(-) diff --git a/.zenodo.json b/.zenodo.json index 3b0bccf..65ae3f0 100644 --- a/.zenodo.json +++ b/.zenodo.json @@ -11,7 +11,7 @@ "affiliation": "Independent researcher" } ], - "description": "

webgpu-fly runs a whole-animal Drosophila nervous system inside a web browser with no installation and no server. The FlyWire FAFB whole-brain connectome (139,255 neurons, ~15 million synaptic connections) and the Janelia MANC ventral-nerve-cord connectome (23,188 neurons, 5.2 million connections) are each simulated as leaky integrate-and-fire (LIF) networks in fused WebGPU compute kernels — gather, integrate, threshold and reset in a single kernel, with presynaptic-neurotransmitter signs pre-baked into the connection weights so the inner loop never branches on excitatory/inhibitory type.

The brain's descending command neurons drive the spinal cord by cell-type name match (the same named cell on both sides of the brain–VNC boundary), and the spine's 369 leg motor neurons are averaged into a walking magnitude and a turn bias that scale a hand-written tripod gait, which in turn actuates a physically simulated 67-body, 111-actuator TuragaLab flybody model running in MuJoCo compiled to WebAssembly. The connectome scales that gait; it does not generate the stepping rhythm, which is an analytic sinusoid of simulation time. A 64×16 retina rendered each frame from the fly's own head pose is fed back into the brain's optic neurons, closing a sensorimotor loop. An optional trained reinforcement-learning walking policy (Vaxenburg et al. 2025) runs as a pure-TypeScript forward pass and walks the body from leg actuation and ground reaction alone, with the kinematic assist switched off: 2.004–2.021 cm per simulated second against a 2.0 cm/s command, uprightness +0.997, no capsize across three repetitions, versus 0.032 cm per simulated second with the policy disabled. That path bypasses the brain and the ventral nerve cord entirely — it is the published policy walking the fly, not the connectome. The forward pass is checked element-wise against an independent NumPy re-run of the same extracted weights; that check validates the port's arithmetic, not the assumed layer architecture against the original SavedModel.

The deployment is a game: the player fires real descending neurons with keypresses to steer the fly to a target, and a winning run produces a deterministic, shareable replay URL that re-executes the identical neuron cascade against the same connectome — a brain trace, not a video. Performance is reported honestly: the brain LIF kernel is memory-bandwidth-bound and runs at ~0.25 kHz of biological time on an Apple M2 Pro, benchmarked on the same machine against NEST 3.10 (0.67 kHz) and a hand-written multicore Rust port (0.45 kHz). The original 1 kHz target was unreachable for any of the three on that hardware; the contribution is reachability — a real connectome simulation behind a single URL — not raw throughput. Known limitations (the connectome scaling rather than generating the gait, the closed-loop visual-reflex approximation, kinematic-assist options, the unverified policy architecture) are enumerated in LIMITATIONS.md.", + "description": "

webgpu-fly runs a whole-animal Drosophila nervous system inside a web browser with no installation and no server. The FlyWire FAFB whole-brain connectome (139,255 neurons, ~15 million synaptic connections) and the Janelia MANC ventral-nerve-cord connectome (23,188 neurons, 5.2 million connections) are each simulated as leaky integrate-and-fire (LIF) networks in fused WebGPU compute kernels — gather, integrate, threshold and reset in a single kernel, with presynaptic-neurotransmitter signs pre-baked into the connection weights so the inner loop never branches on excitatory/inhibitory type.

The brain's descending command neurons drive the spinal cord by cell-type name match (the same named cell on both sides of the brain–VNC boundary), and the spine's 369 leg motor neurons are averaged into a walking magnitude and a turn bias that scale a hand-written tripod gait, which in turn actuates a physically simulated 67-body, 111-actuator TuragaLab flybody model running in MuJoCo compiled to WebAssembly. The connectome scales that gait; it does not generate the stepping rhythm, which is an analytic sinusoid of simulation time. A 64×16 retina rendered each frame from the fly's own head pose is fed back into the brain's optic neurons, closing a sensorimotor loop. An optional trained reinforcement-learning walking policy (Vaxenburg et al. 2025) runs as a pure-TypeScript forward pass and walks the body from leg actuation and ground reaction, with the kinematic assist switched off: 2.019 cm per simulated second against a 2.0 cm/s command, versus 0.032 cm per simulated second with the policy disabled. The forward translation is earned that way; the posture is not. A pitch/roll attitude damper runs every substep, and with it disabled the fly capsizes and stops walking — 0.068 cm per simulated second, uprightness -0.87 — so the +0.997 uprightness and the absence of capsizes across three repetitions are products of that damper, not of the policy. That path bypasses the brain and the ventral nerve cord entirely — it is the published policy walking the fly, not the connectome. The forward pass is checked element-wise against an independent NumPy re-run of the same extracted weights; that check validates the port's arithmetic, not the assumed layer architecture against the original SavedModel.

The deployment is a game: the player fires real descending neurons with keypresses to steer the fly to a target, and a winning run produces a deterministic, shareable replay URL that re-executes the identical neuron cascade against the same connectome — a brain trace, not a video. Performance is reported honestly: the brain LIF kernel is memory-bandwidth-bound and runs at ~0.25 kHz of biological time on an Apple M2 Pro, benchmarked on the same machine against NEST 3.10 (0.67 kHz) and a hand-written multicore Rust port (0.45 kHz). The original 1 kHz target was unreachable for any of the three on that hardware; the contribution is reachability — a real connectome simulation behind a single URL — not raw throughput. Known limitations (the connectome scaling rather than generating the gait, the closed-loop visual-reflex approximation, kinematic-assist options, the pitch/roll attitude damper, the unverified policy architecture) are enumerated in LIMITATIONS.md.", "keywords": [ "WebGPU", "WebAssembly", diff --git a/CITATION.cff b/CITATION.cff index 634e036..218bfae 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -42,11 +42,15 @@ abstract: >- rendered from the fly's head pose feeds back into the brain's optic neurons. An optional trained reinforcement-learning walking policy (Vaxenburg et al. 2025) runs as a pure-TypeScript forward pass and walks - the body from leg actuation and ground reaction alone, with the - kinematic assist off: 2.004-2.021 cm per simulated second against a - 2.0 cm/s command, uprightness +0.997, no capsize across three - repetitions, versus 0.032 cm per simulated second with the policy - disabled. That path bypasses the brain and the ventral nerve cord — it + the body from leg actuation and ground reaction, with the kinematic + assist off: 2.019 cm per simulated second against a 2.0 cm/s command, + versus 0.032 cm per simulated second with the policy disabled. The + forward translation is earned that way; the posture is not. A pitch/roll + attitude damper runs every substep, and with it disabled the fly + capsizes and stops walking — 0.068 cm per simulated second, uprightness + -0.87 — so the +0.997 uprightness and the absence of capsizes are + products of that damper, not of the policy. + That path bypasses the brain and the ventral nerve cord — it is the published policy walking the fly, not the connectome. The forward pass is checked element-wise against an independent NumPy re-run of the same extracted weights, which validates the port's diff --git a/LIMITATIONS.md b/LIMITATIONS.md index ecc6ddf..9753b70 100644 --- a/LIMITATIONS.md +++ b/LIMITATIONS.md @@ -15,9 +15,10 @@ the fly.** Roughly 20.3M connectome edges reach the body as about one scalar magnitude plus a turn bias per tick, and those two numbers scale a hand-written `sin(t × 10 Hz)` tripod. §8 is the complete shortcut inventory, with measured numbers for what the body does once the assist is off. A *published RL policy* -(Vaxenburg et al. 2025) does walk the body under physics with the assist off -(§4.1) — that policy bypasses the brain and the spine entirely, so it is not -the fly's own brain doing the walking either. +(Vaxenburg et al. 2025) does translate the body under physics with the assist +off (§4.1), though it stays upright only because a pitch/roll damper outside +the assist holds it there — and that policy bypasses the brain and the spine +entirely, so it is not the fly's own brain doing the walking either. --- @@ -56,31 +57,69 @@ the fly's own brain doing the walking either. simply out of scope for v1. Changing the confidence threshold requires rebuilding `brain.bin`. -## 3. Dynamics validation is qualitative, not quantitative +## 3. Dynamics validation is qualitative — and calibrated, not independent - We check the **shape** of the dynamics: no-input networks go silent (no - runaway), Kenyon cells fire sparsely (~5–15%) under sensory drive, - consistent with Shiu et al. 2024. + runaway), Kenyon cells fire sparsely (~5–15%) under sensory drive. +- **`w_syn` is not taken from Shiu et al. — it is fitted to the number we then + report.** Our own source names their free parameter as 0.275 mV per synapse + (`src/sim.ts:16`); we ship 0.005 (`src/sim.ts:39`), and the comment above it + says why in as many words (`src/sim.ts:25-30`): + + > w_syn is tuned EMPIRICALLY (not from peak-matching) to land KC at the + > canonical 5-15% on Mixed sensory. Alpha synapse integrates each spike + > over ~5ms so the cascade amplifies non-linearly vs old single-step + > direct injection — peak-matching gives way too hot a brain (73% KC). + > 0.005 keeps the dataset's natural cascade strength visible without + > runaway. + + So KC sparsity is a **calibration target, not an independent validation**: + the single free synaptic weight was tuned until the sparsity landed in the + canonical band, and that agreement is then reported as a result. What the + check actually shows is that a `w_syn` exists which puts this network in a + plausible regime — sparse rather than silent or saturated — not that the + network reproduces Shiu et al.'s dynamics. **The sparsity number is not + independent evidence.** - We have **not** done a quantitative, cell-type-resolved firing-rate match - against a published reference simulation across the whole brain. The - sparsity check is the strongest dynamics claim we stand behind. + against a published reference simulation across the whole brain. With + `w_syn` fitted to the sparsity band, there is no check here that is both + quantitative and independent of that fit. ## 4. The brain → spine → body path has documented approximations These are the "honest gaps" from the README, restated as limitations. §8 is the complete list; these four are the ones with the longest history. -1. **Trained RL walker walks — after four port fixes.** With the kinematic - assist explicitly off, the policy drives the body from actuator → ground - reaction alone: **2.004–2.021 cm per simulated second** against a 2.0 cm/s - command, 5.03–5.08 cm of travel per window, uprightness **+0.997** at the - end of the window, and **no capsize in 3 of 3 reps**, with |action| max - ≈ 5.9 while upright — inside flybody's native band (~6). Control condition, +1. **Trained RL walker translates — after four port fixes, and only with the + attitude damper on.** With the kinematic assist explicitly off and the + damper in its shipped on state, nothing on this path writes the body's + translational velocity (`qvel[0..2]`), so the forward motion is genuinely + actuator → ground reaction: **2.004–2.021 cm per simulated second** against + a 2.0 cm/s command, 5.03–5.08 cm of travel per window, uprightness + **+0.997** at the end of the window, and **no capsize in 3 of 3 reps**, with + |action| max ≈ 5.9 while upright — inside flybody's native band (~6). Control condition, policy never enabled and assist still off: 0.032 cm/sim s, uprightness - 0.999 — the fly just stands there, so the locomotion comes from the policy + 0.999 — the fly just stands there, so the translation comes from the policy and not from anything else. In the e2e suite (assist at its default) the walker's displacement is dx = +1.642 cm, previously −1.174 cm. Reproduce - with `.walkbench.mjs`. + with `tools/walkbench.mjs`. + + **The uprightness is not the policy's.** The pitch/roll damper (§8) is a + separate intervention that is not gated by the assist, and it is what keeps + this path on its feet. Measured A/B, trained policy, assist off, 3 reps + each: + + | Pitch/roll damper | cm/sim s | Uprightness at end | Min uprightness | + |---|---|---|---| + | **ON** (as shipped) | 2.019 | **+0.997** | +0.997 | + | **OFF** | 0.068 | **−0.87** | −0.913 | + + Commanded speed is 2.0 cm/s in both. With the damper off the fly capsizes + and stops walking — 0.068 cm/sim s is closer to the 0.032 cm/sim s of a + body with no controller at all (damper on) than to the commanded 2.0. So + the split is: **translation is earned, attitude is not.** A sentence of the + form "walks from leg actuation and ground reaction alone" is false about + this path; the honest version keeps the two halves apart. Until today this item read "does not walk", and it was accurate: with the assist off the fly capsized within ~1.5 s and stayed on its back @@ -107,9 +146,9 @@ the complete list; these four are the ones with the longest history. 59-dim `actuator_activation` observation real instead of all zeros and softens the plant ~5.5× per control tick. - **What this does not change:** the pitch/roll damper (§8) is not gated by - the assist and runs on this path too, so the uprightness numbers above are - not a damper-free result; the forward pass has still never been compared + **What this does not change:** the uprightness numbers above are not a + damper-free result, and the A/B says what the damper-free result is; the + forward pass has still never been compared against the published SavedModel (see the end of §8); and this is still not the connectome walking the fly — it is a published RL policy on a path that bypasses the brain and the spine. @@ -142,7 +181,7 @@ the complete list; these four are the ones with the longest history. synthetic, not a training clip. Real fly mocap from the Vaxenburg deposit is wired in as opt-in (`__walkingRefFromMocap`), and that branch is **not** fixed: it is baked at 50 Hz (`tools/bake_walking_ref.py:46`) and replayed - one frame per 2 ms control tick (`src/physics.ts:971`), a 10× rate error, + one frame per 2 ms control tick (`src/physics.ts:978`), a 10× rate error, and the 65-frame lookahead exceeds the 57-frame trajectory so the window wraps mid-observation. Measured before the item-1 fixes: enabling it takes |action| max from ~7 to 3097–3250 and the fly spins 944–1051° in ~1.8 @@ -180,15 +219,18 @@ the complete list; these four are the ones with the longest history. ## 8. Full shortcut inventory — what actually moves the body -The "Honest mode" button flips exactly three flags (`src/main.ts:669-683`). -The table below has thirteen rows, and nine of them are behind no toggle at -all. This section is all of them — the ones the button covers and the ones it -does not — so the button is not the only place they are disclosed. +The "Honest mode" button flips exactly three flags (`src/main.ts:772-774`). +The table below has fourteen rows, and ten of them are marked **no** — outside +the button entirely. This section is all of them — the ones the button covers +and the ones it does not — so the button is not the only place they are +disclosed. -Nothing here is a claim about the neural simulation. The FlyWire and MANC -connectomes are real, both LIF networks genuinely run on the GPU, and the -stimulus→cascade dynamics are connectome-derived. What follows is about how -the *body* is driven, which is a different and much weaker story. +Almost nothing here is a claim about the neural simulation. The FlyWire and +MANC connectomes are real, both LIF networks genuinely run on the GPU, and the +stimulus→cascade dynamics are connectome-derived. What follows is mostly about +how the *body* is driven, which is a different and much weaker story — one +exception is the deafferentation row, which is about what the cord is never +told. ### The information bottleneck @@ -204,7 +246,7 @@ MANC 23,188 neurons / 5,243,574 edges ↓ and the direction sign is discarded — it comes from the hand-wired 200-neuron synthetic spine main.ts:470 ↓ driveLegs(walk, turn) room.ts:640 - = sin(t · 10 Hz), 18 of 48 leg actuators physics.ts:663-719 + = sin(t · 10 Hz), 18 of 48 leg actuators physics.ts:734-790 ``` **Roughly 20.3 million connectome edges reach the body as about one scalar @@ -220,17 +262,31 @@ geometry (`src/vnc.ts:369-384`). |---|---|---|---| | **Kinematic assist** — writes freejoint `qvel[0]`, `qvel[1]`, `qvel[5]` from `fwdCmd`/`turnCmd`, re-asserted every substep before `mj_step`. Still what moves the body on the **CPG path**; the trained-policy path no longer needs it (§4.1) | ground reaction from leg contact | **yes** (`Physics.kinematicAssistEnabled`) | `src/physics.ts` `step()`, default on at `Physics.kinematicAssistEnabled` | | **Stale CPG command under the policy — resolved** — `fwdCmd`/`turnCmd` are written only by `driveLegs` and the policy path skips `driveLegs`; nothing used to zero them, so the last CPG command kept driving the body throughout "trained walking". The policy path now clears the stale command on takeover | — | n/a — no longer a live shortcut | `src/physics.ts`, `src/room.ts:633-647` | -| **Pitch/roll attitude damper** — `qvel[3] *= 0.85; qvel[4] *= 0.85` per substep, ×0.039 per 2 ms control tick | balance, and the body's ability to tip at all | **no** — it sits before and outside the assist guard | `src/physics.ts:656-659` | +| **Pitch/roll attitude damper** — `qvel[3] *= 0.85; qvel[4] *= 0.85` per substep, ×0.039 per 2 ms control tick. It is what keeps the trained policy upright: damper off, the policy capsizes and drops from 2.019 to 0.068 cm/sim s (§4.1) | balance, and the body's ability to tip at all | **no** — it sits before and outside the assist guard, and the button does not flip it. It is now gated by its own flag (`Physics.attitudeDamperEnabled`, default on) so it can be measured | `src/physics.ts:625`, `:663-666` | | **Boot stimulus drives itself** — science mode auto-runs `STIMULI[0]` at load, saturating the spine to `fwdCmd = 0.99999` for the length of its window; the drive is put back to rest when that window ends, so the residual no longer survives to the first user click. `decayDrive()` is defined and never called (one grep hit, the definition) | a brain whose drive responds to what you click | **no** | `src/main.ts:1284-1288`, `src/main.ts:499` | -| **Tripod CPG is the source of leg timing** — `phase = data.time · 10 Hz`, hard-coded gait constants, 3 of 8 DOFs driven per leg | motor-neuron output setting stance/swing | **no** | `src/physics.ts:718-725`, `:727-783`, actuator cache `:277-284` | +| **The cord is deafferented** — every one of MANC's 6,282 sensory neurons receives zero input. The only drive written into the VNC is the 7 DN types' brain rate; the rest of the `ext` vector stays zero (`src/main.ts:438-452`), and the offline rhythm script drives it the same way (`tools/vnc_rhythm.py:436-437`). Count from `public/vnc.meta.json` (`cell_classes.sensory`) | load, campaniform, hair-plate and chordotonal feedback — which is load-bearing for real insect leg coordination | **no** | `src/main.ts:438-452`, `tools/vnc_rhythm.py:436-437` | +| **Tripod CPG is the source of leg timing** — `phase = data.time · 10 Hz`, hard-coded gait constants, 3 of 8 DOFs driven per leg | motor-neuron output setting stance/swing | **no** | `src/physics.ts:725-732`, `:734-790`, actuator cache `:277-284` | | **Wing motion is hand-written** — 218 Hz analytic stroke, amplitude hard-capped at ×0.2 of flybody's canonical pattern because anything above ~0.25 launches the freejoint body | wing motor neurons (MANC's 66 are read for the readout only) | **no** | `src/physics.ts:558-582`, cap at `:568` | -| **`jumpImpulse` writes `qvel[2]` directly** | leg extension producing a takeoff | **no** | `src/physics.ts:1063-1065`, called from `src/main.ts:495` | -| **Adhesion clamped to 1.0** at init and whenever walk drive < 0.01 — a standing fly is glued to the floor | claw contact and friction holding a stationary fly | **no** | `src/physics.ts:285-291`, `:773-776` | +| **`jumpImpulse` writes `qvel[2]` directly** | leg extension producing a takeoff | **no** | `src/physics.ts:1074-1076`, called from `src/main.ts:495` | +| **Adhesion clamped to 1.0** at init and whenever walk drive < 0.01 — a standing fly is glued to the floor | claw contact and friction holding a stationary fly | **no** | `src/physics.ts:287-293`, `:780-783` | | **Visual-reflex angle bypass** — `turn ∝ retinal angle`, forward speed from retinal area | the brain's optic→DN contralateral cascade | **yes**, but the brain path falls back to the identical law when cascade asymmetry < 0.05, and the code records the cascade's sign as empirically **wrong** for tracking | `src/vnc.ts:369-378`; brain path `:352-368`; sign note `:322-326` | | **Sweep-mode spine bypass** — target lost for 4+ ticks writes a scripted alternating scan turn straight to the body | search behaviour emerging from the brain | **no** | `src/main.ts:998-1005` | -| **Walking reference** — synthetic world-space trajectory by default, now closed-loop against the live body pose (§4.1), but still synthetic rather than a training clip; the mocap opt-in is **still** baked at 50 Hz and replayed at 500 Hz, and the 65-frame lookahead still exceeds the 57-frame clip | the policy's training reference clip | switches to mocap, which measures **worse** (§4.4) | `src/physics.ts:974-1000`, `:971`; `tools/bake_walking_ref.py:46` | +| **Walking reference** — synthetic world-space trajectory by default, now closed-loop against the live body pose (§4.1), but still synthetic rather than a training clip; the mocap opt-in is **still** baked at 50 Hz and replayed at 500 Hz, and the 65-frame lookahead still exceeds the 57-frame clip | the policy's training reference clip | switches to mocap, which measures **worse** (§4.4) | `src/physics.ts:981-1008`, `:978`; `tools/bake_walking_ref.py:46` | | **"Evolve gait (WebGPU ARS)" does not evolve against MuJoCo** — the fitness is a 1-D point-mass rollout with analytic thrust and quadratic drag: no gravity, no ground contact, no body — and the winner is written into the live physics body | optimizing the gait against the actual simulated fly | **no** | `src/shaders/evolve.wgsl:4-6`, `:101-104`; applied at `src/main.ts:1054-1056` | -| **The speed readout displays the assist** — `bodySpeed` reads `qvel[0..1]`, the exact slots the assist writes immediately before `mj_step` | measured locomotion | **no** | `src/physics.ts:1069-1073`, rendered at `src/main.ts:739,746` | +| **The speed readout displays the assist** — `bodySpeed` reads `qvel[0..1]`, the exact slots the assist writes immediately before `mj_step` | measured locomotion | **no** | `src/physics.ts:1079-1083`, rendered at `src/main.ts:739,746` | + +The deafferentation row also scopes what `tools/vnc_rhythm.py` can and cannot +conclude. Any negative rhythm result it produces is a finding about **our +deafferented LIF model of MANC**, not about the MANC connectome. Per-neuron +state in the kernel is membrane voltage, a refractory counter and a two-state +alpha synapse — four read-write buffers besides the spike bitmask +(`src/shaders/lif.wgsl:36-37`, `:39-40`) — with no spike-frequency adaptation, no synaptic depression, no +conduction delay, no rebound current, and, per that row, no sensory feedback +into the cord. The mechanisms a half-centre oscillator relies on to terminate a +burst are therefore absent by construction, and a network built like this +failing to alternate is a property of the reduction before it is evidence about +the wiring. Read such a result as "this model does not oscillate," never as +"MANC has no CPG." One more, about evidence rather than physics: **the walking policy's forward pass is not verified against the published SavedModel.** @@ -301,11 +357,15 @@ Reading it: significant figures, because the other two flags are only read in code paths CPG mode never enters. -The pitch/roll damper is not in that table, because **the project has never -been run with it off.** It has been ×0.005 per CPG render frame for the entire -life of the codebase, in every mode including Honest mode, so no measurement -here — or in any commit message — describes a fly that could tip over. That -baseline is unmeasured. +The pitch/roll damper is not a column in that table, because every one of those +16 runs had it on. It was ×0.005 per CPG render frame in every mode including +Honest mode, and it was not switchable, so **no uprightness figure recorded +anywhere before the `Physics.attitudeDamperEnabled` experiment is damper-free** +— not in this table, not in §4.1's original numbers, not in any commit message. +The damper-off baseline is no longer unmeasured, but it has only been measured +on one path: the trained policy with the assist off (§4.1), where turning the +damper off takes the fly from 2.019 to 0.068 cm/sim s and from +0.997 +uprightness to −0.87. The CPG path has still never been run damper-free. --- diff --git a/README.md b/README.md index 22952c4..9fc4571 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ approximation, and every approximation and shortcut is inventoried in - **Not a scientific simulator replacement.** NEST / Brian2 / NEURON are faster, biophysically detailed, and validated. For real fly-brain dynamics research, use those. - **Not biophysically detailed.** Neurons are LIF with a two-state alpha synapse — no ion channels, dendritic compartments, or neuromodulation. -- **Not quantitatively validated** whole-brain. The dynamics check is qualitative (Kenyon-cell sparsity matches Shiu et al. 2024), not a cell-type-resolved rate match. +- **Not quantitatively validated** whole-brain. The one dynamics check is Kenyon-cell sparsity in the canonical 5–15% band — and `w_syn` was tuned to land it there (`src/sim.ts:25-30`), so it is a calibration target, not an independent validation of Shiu et al. 2024. - **Not faster than the reference.** It runs *slower* than real time. The win is reachability, not throughput. See [`LIMITATIONS.md`](./LIMITATIONS.md). @@ -107,7 +107,7 @@ mode, ARS evolver, raw spike-rate log. | **Spine** | [Janelia MANC](https://www.janelia.org/project-team/flyem/manc-connectome) connectome (Takemura et al. 2024) | 23,188 VNC neurons, 5.2M edges, second WebGPU LIF instance | | **Body** | [TuragaLab/flybody](https://github.com/TuragaLab/flybody) MJCF (Vaxenburg et al. 2025, *Nature*) | 67 bodies, 111 actuators, real physics in MuJoCo/WASM | | **Eyes** | offscreen render-to-texture from fly head pose | 64×16 retinal sample fed to brain optic neurons | -| **Walker** | trained RL policy ([Vaxenburg et al. 2025 Figshare](https://janelia.figshare.com/articles/dataset/25309105)) | LayerNormMLP, 741-dim obs → 59 actions, pure-TS forward pass. Walks the body under physics with the kinematic assist **off** — 2.004–2.021 cm per simulated second against a 2.0 cm/s command, upright +0.997, no capsize in 3/3 reps. Checked element-wise against a numpy re-run of the same extracted weights (`tools/verify_walking_policy.py`) — that validates the port's arithmetic, not the assumed architecture against the original SavedModel | +| **Walker** | trained RL policy ([Vaxenburg et al. 2025 Figshare](https://janelia.figshare.com/articles/dataset/25309105)) | LayerNormMLP, 741-dim obs → 59 actions, pure-TS forward pass. Translates the body under physics with the kinematic assist **off** — 2.004–2.021 cm per simulated second against a 2.0 cm/s command. It stays upright (+0.997) only with the pitch/roll damper on; with the damper off it capsizes and covers 0.068 cm/sim s. Checked element-wise against a numpy re-run of the same extracted weights (`tools/verify_walking_policy.py`) — that validates the port's arithmetic, not the assumed architecture against the original SavedModel | Brain → spine wiring is by **cell-type name match** (`DNa01` in the brain is the same neuron as `DNa01` in the VNC — brain side has the soma, VNC side the axon). @@ -123,14 +123,20 @@ the leg phase itself is `sin(sim_time · freq)`. (Caveat: it's a name join acros connectomes, not a reconstructed synaptic bridge — see [`LIMITATIONS.md`](./LIMITATIONS.md) §5.) -The **trained RL walking policy** is a separate path, and it does walk the body -from leg actuation and ground reaction alone: with the kinematic assist off it -covers 2.004–2.021 cm per simulated second against a 2.0 cm/s command and stays -upright (+0.997, no capsize in 3/3 reps), while a run with the policy never -enabled travels 0.032 cm/sim s. That path bypasses the brain and the spine -entirely — it is Vaxenburg et al.'s published policy walking the fly, not the -connectome. [`LIMITATIONS.md`](./LIMITATIONS.md) §4.1 has the numbers and the -four port defects that had to be fixed to get there. +The **trained RL walking policy** is a separate path, and its *translation* is +earned: with the kinematic assist off nothing on that path writes the body's +translational velocity, so the 2.004–2.021 cm per simulated second it covers +against a 2.0 cm/s command comes from leg actuation and ground reaction, while a +run with the policy never enabled travels 0.032 cm/sim s. Its *attitude* is not +earned. A pitch/roll damper that sits outside the assist multiplies the body's +pitch and roll angular velocity by 0.85 every substep (`src/physics.ts:663-666`, +×0.039 per control tick); with that damper off the same policy capsizes and +covers 0.068 cm/sim s. So the +0.997 uprightness is the damper's doing, not the +policy's. +That path also bypasses the brain and the spine entirely — it is Vaxenburg et +al.'s published policy walking the fly, not the connectome. +[`LIMITATIONS.md`](./LIMITATIONS.md) §4.1 has the damper A/B, the numbers, and +the four port defects that had to be fixed to get there. --- diff --git a/index.html b/index.html index dc21784..ed2afd2 100644 --- a/index.html +++ b/index.html @@ -471,21 +471,34 @@

Four real datasets, wired together.

- The brain and spine are joined the way real biology does it: a command - neuron named DNa01 in the brain is the same cell as - DNa01 in the spinal cord. Nothing in that path is learned - or scripted — it is wire for wire from the real animal. Past the - spinal cord it changes: the motor neurons set a walking speed and a - turn, and a hand-written leg rhythm does the actual stepping. + The brain and the spine are joined by name, not by wire: a + command neuron called DNa01 in the brain is matched to the + neuron with the same name in the spinal cord. Biology motivates the + match — that cell really does keep its body in the brain and send its + axon down the cord — but the two maps come from two different flies (a + female brain, a male nerve cord), and no reconstructed connection + crosses between them. What crosses is a number: the brain neuron's + recent firing rate, averaged over its left and right copies and scaled + by a constant we picked, dropped into the spinal neurons that share its + name. The spiking on either side is real; the handoff is ours. Past the + spinal cord it changes again: the motor neurons set a walking speed and + a turn, the choice of forward or backward comes from a small hand-wired + spine rather than from the real one, and a hand-written leg rhythm does + the actual stepping.

There is also a second mode that hands the legs to a - trained walking policy published with the body model. That one - really does walk the fly on its own legs, at the speed you ask it - for, with the push we normally give the body switched off. It is - worth being clear about what it is: a neural network someone trained - with reinforcement learning, not the fly's own brain — the connectome - plays no part in it. + trained walking policy published with the body model. With the + push we normally give the body switched off, the forward motion is + genuinely the legs' doing — the fly covers 2.019 cm per simulated + second against a 2.0 cm/s command, pushing off the ground. Staying + upright is not: a stabiliser bleeds off the fly's pitch and roll on + every physics substep, and with that switched off too it topples over + and gets nowhere (0.068 cm per simulated second). The forward motion + is earned; the balance is propped up. It is also worth being clear + about what the policy is: a neural network someone trained with + reinforcement learning, not the fly's own brain — the connectome plays + no part in it.

diff --git a/package.json b/package.json index 93b4096..9ab1a01 100644 --- a/package.json +++ b/package.json @@ -23,7 +23,7 @@ "bench:brain": "playwright test tests/bench.spec.ts --reporter=list", "build:slim": "npm run build && rm -rf dist/flybody dist/flybody.bundle.bin dist/brain.bin dist/brain.meta.json dist/vnc.bin dist/vnc.meta.json dist/walking-policy.bin dist/walking-obs-norm.bin dist/walking-ref.bin dist/walking-policy-fixtures.json", "deploy": "npm run build:slim && npx --yes wrangler pages deploy dist --project-name=webgpu-fly --branch=main", - "deploy:hf": "npm run build:slim && cp space/README.md space/.gitattributes dist/ && rm -f dist/_headers && hf upload abgunaydin/webgpu-fly dist . --repo-type=space", + "deploy:hf": "npm run build:slim && cp space/README.md space/.gitattributes NOTICE LICENSE LICENSE-FLYBODY LIMITATIONS.md dist/ && rm -f dist/_headers && hf upload abgunaydin/webgpu-fly dist . --repo-type=space", "deploy:vercel": "npm run build && vercel --prod", "deploy:r2": "bash tools/upload_to_r2.sh" }, diff --git a/space/README.md b/space/README.md index 98a3407..af4cbbf 100644 --- a/space/README.md +++ b/space/README.md @@ -21,8 +21,13 @@ runs in the tab.** Two separate things move the body, and the difference matters: - A **trained RL walking policy** (Vaxenburg et al. 2025) genuinely walks the - fly from leg actuation and ground reaction alone. That path bypasses the - brain and the spine entirely. + fly forward from leg actuation and ground reaction — 2.019 cm per simulated + second against a 2.0 cm/s command, with the kinematic assist off. It does + not keep the fly upright, though: a pitch/roll damper bleeds off the body's + pitch and roll angular velocity every substep, and with that damper off the + fly capsizes and stops walking (0.068 cm/sim s, uprightness −0.87). The + translation is earned; the posture is not. That path bypasses the brain and + the spine entirely. - The **connectome** drives a hand-written tripod gait. It scales that gait but does not generate its rhythm — the leg phase is `sin(sim_time · freq)`. @@ -49,16 +54,21 @@ so the 314 MB cache may not survive between visits. ## Credits and licensing -Code is MIT. The data is not, and each piece keeps its own terms: +The `license: mit` in this Space's header describes the source code only. The +~314 MB of data the app downloads is not MIT, and each piece keeps its own +terms: | | | |---|---| -| Brain connectome | [FlyWire](https://flywire.ai) FAFB, CC-BY | -| Ventral nerve cord | [Janelia MANC](https://www.janelia.org/project-team/flyem/manc-connectome) (Takemura et al. 2024) | -| Body model + walking policy | [TuragaLab/flybody](https://github.com/TuragaLab/flybody) (Vaxenburg et al. 2025), Apache-2.0 | -| Physics | MuJoCo compiled to WebAssembly | +| Brain connectome | [FlyWire](https://flywire.ai) FAFB, CC-BY 4.0 | +| Ventral nerve cord | [Janelia MANC](https://www.janelia.org/project-team/flyem/manc-connectome) (Takemura et al. 2024), CC-BY 4.0 | +| Body model (MJCF + meshes) | [TuragaLab/flybody](https://github.com/TuragaLab/flybody) (Vaxenburg et al. 2025), Apache-2.0 | +| Walking policy | [Janelia Figshare deposit](https://janelia.figshare.com/articles/dataset/25309105) (Vaxenburg et al. 2025), CC-BY 4.0 | +| Physics | MuJoCo compiled to WebAssembly, Apache-2.0 | -Full attribution in `NOTICE`; every approximation and shortcut is inventoried -in `LIMITATIONS.md`. +Full attribution in [NOTICE](NOTICE); the Apache-2.0 text is in +[LICENSE-FLYBODY](LICENSE-FLYBODY) and the MIT text in [LICENSE](LICENSE). +Every approximation and shortcut is inventoried in +[LIMITATIONS.md](LIMITATIONS.md). Source: diff --git a/src/physics.ts b/src/physics.ts index 0ba3b51..53c6327 100644 --- a/src/physics.ts +++ b/src/physics.ts @@ -616,6 +616,14 @@ export class Physics { * path is the drive scalar, not locomotion. */ static kinematicAssistEnabled = true; + /** Pitch/roll angular-velocity damper applied every substep before + * mj_step (x0.85, i.e. x0.039 over a 20-substep control tick). It is a + * separate intervention from the kinematic assist and, unlike it, has + * been on unconditionally for the project's whole life — so no + * uprightness number recorded before this flag existed is damper-free. + * Exposed so that can be measured rather than assumed. */ + static attitudeDamperEnabled = true; + /** Step physics N times. * * Leg and wing actuators, contacts and ground reaction are real @@ -627,9 +635,9 @@ export class Physics { * a 32-substep render frame). This is NOT gated by * kinematicAssistEnabled: it runs in honest mode and under the * trained policy. Leg and gravity torques about those two axes are - * largely absorbed instead of integrated. It has never been run in - * the off state, so the undamped baseline is unmeasured — do not - * assume the fly stands up without it. + * largely absorbed instead of integrated. Measured through + * attitudeDamperEnabled: off, the trained policy goes from 2.019 + * to 0.068 cm/sim s and from +0.997 uprightness to -0.87. * * 2. Kinematic assist — when kinematicAssistEnabled and a drive * command is set, translation (qvel[0], qvel[1]) and yaw (qvel[5]) @@ -652,7 +660,7 @@ export class Physics { for (let s = 0; s < substeps; s++) { const qpos = this.data.qpos as Float64Array; const qvel = this.data.qvel as Float64Array; - if (qvel && qvel.length >= 6) { + if (Physics.attitudeDamperEnabled && qvel && qvel.length >= 6) { qvel[3] *= 0.85; // pitch damping qvel[4] *= 0.85; // roll damping }