Repository navigation
refactor(gemma4): one forward path carries the numeric gates - #1015
Conversation
6bea7ad to
7d3ee9a
Compare
Signed-off-by: Feathbow <feathbow@gmail.com>
7d3ee9a to
4ead109
Compare
|
The strict gate has its exact-head runner output. At the rebased head the reworked gate already ran with the strict assert and passed under the default bf16 profile: thirteen compared rows, zero argmax flips, per-position max |dlogit| 0 to 1.5 against the 2.0 ceiling. Under The body and Issue text move with it: the short trajectory itself never crosses the window and says so, the crossing stays pinned by the waypoint, mixed-window, overlap and ragged gates, the no-coverage claim contracts to what the gate uniquely adds (full-vocabulary drift at every position of the nine-token prompt plus four non-cascading continuation steps), and the near-tie description is gone. |
|
A rule this PR introduced cannot hold, and it has now been caught: The same five ids and the same tail; the top two 0.0625 apart on one side, 0.1875 on the other, the rows agreeing to 0.3125 elsewhere — reduction-order drift across a near tie, not a moved distribution. Green on boxes where that pair happens to sit wider apart, so the rule fails by environment, not by behaviour. The other gates this PR touched have since moved to the near-tie form ( |
Description
Closes #1014
The serving slices already retired the probe forward: the HF fixture gates drive the serving path teacher forced, with the same fixtures, tolerances and distributional floors. What remained of the old arrangement goes here. An incremental-versus-recompute gate answers for the one path's self-consistency where the probe comparison used to: every prompt position's logits through the decode arena match a whole-prompt recompute, and four decode steps fed the recompute's own picks match it too. The release-freezing test switch retires with the eviction gate it served —
window_leftmasks out-of-window keys in-kernel, the waypoint gates cross the window against HF with the resident-page arithmetic asserted, and both arms of the new gate release their fronts as production does.Test Env
Single GPU (
sm_89, 48 GiB, x86_64)Verification
cargo fmt --all -- --checkandgit diff --checkagainstmainpass; the Gemma 4 release server builds; all-target Gemma Clippy passes with-D warnings; release lib tests pass.incremental_serving_matches_recomputepasses 1/1 through the runner (whose membership check owns the manifest swap), on the golden fixture's short prompt the 2.0 ceiling was calibrated on: per-position max |dlogit| runs 0 to 0.79 against the 2.0 ceiling, with one near-tie top-1 flip admitted under the rule that both rows must rank the two candidates within the ceiling of each other; the four decode steps stay under it too. The 12B waypoint and greedy HF gates pass unchanged with the same fixtures and tolerances./v1/completionsrequests (31-, 35-, 33- and 1421-token prompts, 64 forced tokens) with bytes identical tomain's binary, md5219de1e3c09c(everything removed was#[cfg(test)]).