Skip to content

profile: register the pool-page switches for Dory, Gemma 4, Spark2.5 and Nemotron's Metal paging - #2306

Open
devopsfortunato wants to merge 1 commit into
ashhart:mainfrom
devopsfortunato:profile-pool-page-switches-1660
Open

devopsfortunato wants to merge 1 commit into
ashhart:mainfrom
devopsfortunato:profile-pool-page-switches-1660

Conversation

@devopsfortunato

@devopsfortunato devopsfortunato commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

The switch registry grows by four rows and their readers route through the registry's environment-first helper, so tensorfold capabilities --json and the startup line cover them: TENSORFOLD_DORY_KV_PAGES (Dory on CUDA, opt-in), TENSORFOLD_GEMMA4_KV_PAGES and TENSORFOLD_SPARK25_KV_PAGES (default-on paging, exact 0 opts out), and TF_NEMOTRON_METAL_KV_PAGES (Nemotron-H on Metal, exact 1), the registry's first metal-backend row. With a variable unset, behavior is identical; with it set, the value reaches the family byte for byte, exactly as std.c.getenv returned before. One new syntax (not_zero) covers Dory's on-unless-exactly-0 reader; the two 0-opt-out rows reuse the existing report_values. docs/profiles.md lists the new coverage.

Checks on this head

  • Head commit: 109fb38 (pushed as profile-pool-page-switches-1660)
  • Lean checker, python3 tools/zig/lean_check.py, on the base (691fb43) and on this head: 5624 files / 325 problems both, lean_new 0 new findings.
  • Tests run, with the command and pass, fail and skip counts:
    • zig build -Dfixed-cpu test-profile: pass.
    • zig build -Dfixed-cpu test-profile-line-host: pass.
    • zig build test --test-timeout 10m (what CI's zig job runs): green in CI on this head, and run locally as the full host suite on one aarch64 host (20 cores, Zig 0.17.0 from .zig-version, an empty Zig cache): exit 0, Build Summary: 569/569 steps succeeded; 7345/7406 tests passed (61 skipped), 4m57s wall.
    • The earlier roce_inflight_test failure was this 4-core Pi's madvise(MADV_NOHUGEPAGE) → EINVAL, environmental and unrelated to this diff; on the host above it passes, and the whole suite is green there.
  • CI: green — all 7 checks success on 109fb38: benchmark-clients ubuntu and windows, fmt, release-paths, zig, zig-macos-dev-tools (compiles the Metal engine module, including the kv_pages.zig edit), zig-windows.
  • Not run, and why: Metal builds and zig build kernel-contract -Dcpu=apple_m1 — no macOS host available here; the Metal-path edit is one registry lookup in nemotron/kv_pages.zig mirroring glm/cuda_kv_host.zig's existing @import("profile") pattern (the build graph injects the registry into every engine module; zig ast-check clean). CUDA device checks and the families' tf-cuda-test *-pages receipts — no GPU on this host; the diff cannot change device behavior: it only registers metadata and preserves every lookup byte for byte.

Receipt

n/a — the change cannot reach inference on its own: it registers existing opt-in switches and preserves every environment lookup byte for byte, unset stays unset. The switches' own identical-token receipts are their landing pull requests: #1284, #1619, #1653 and #1537.

Checklist

  • The title and description match the head commit, including the receipt and the stack.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency: the new registry and profile-line tests do not compile against main (the switches do not exist there) and pass on this head; the scope-aware backend assertions fail against main's registry (its metal-less rows trip them) and pass here.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @devopsfortunato. This reads right: each of the four switches has one reader on main, each moves to
profile.get with the same rule (Dory and Nemotron on Metal off when unset, Gemma 4 and Spark2.5 off only for exact
0), and the four rows are identical, so the disk-cache identity doesn't change either. Before it queues:

  1. A rebase. docs/profiles.md (main's paragraph now says "eight variables") and server/profile_line.zig
    (main's test environment has a TF_GLM_KV_SHARE line) conflict. The rest applies cleanly.
  2. A CHANGELOG line, citing "(profile: register the pool-page switches for Dory, Gemma 4, Spark2.5 and Nemotron's Metal paging #2306, part of RFC: --profile stable | fast-exact | fast, one table for the opt-in switches #1660)": the four switches, startup and capabilities reporting
    them, values read as before and every default unchanged.
    Three small things while you're there: the first registry test's backend list drops main's hip, so please keep it;
    the registry has 14 names with this PR, so the docs' count should say 14 or go; and one sentence on not_zero beside
    the auto and report_values notes would help. And please post the full host suite on the rebased head when it
    lands.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

A correction to our note above, @devopsfortunato: the two conflicts we named came from reading against a main from before your base. a189233f8 already has #2211, which retires TF_GLM_KV_SHARE, so there's nothing to rebase for them, and "eleven variables" is right by that sentence's count.

Still wanted:

  • the CHANGELOG line;
  • hip back in the first test's backend list;
  • a sentence on not_zero;
  • the full host suite on this head.

@devopsfortunato
devopsfortunato force-pushed the profile-pool-page-switches-1660 branch 2 times, most recently from 81be32d to b0f8863 Compare November 6, 2026 09:25
…and Nemotron's Metal paging

The switch registry grows by four rows (ashhart#1660's next group, all measured identical):
TENSORFOLD_DORY_KV_PAGES (opt-in, ashhart#1284), TENSORFOLD_GEMMA4_KV_PAGES and
TENSORFOLD_SPARK25_KV_PAGES (default-on, exact 0 opts out, ashhart#1619, ashhart#1653), and
TF_NEMOTRON_METAL_KV_PAGES, the first metal-backend row (exact 1, ashhart#1537). A new
.not_zero syntax covers Dory's on-unless-0 reader; the 0-opt-out rows reuse
report_values. The four readers go through the registry's environment-first
helper, so an unset variable and every set value behave byte for byte as before.
@devopsfortunato
devopsfortunato force-pushed the profile-pool-page-switches-1660 branch from b0f8863 to 109fb38 Compare November 6, 2026 09:46
@devopsfortunato

Copy link
Copy Markdown
Contributor Author

Thanks @ashhart - all five points are in, on head 109fb3893 (one commit, rebased onto 691fb4334; main moved again while the checks ran, so this is the second rebase).

1. The rebase. Done twice: first onto 34d42fcc8, then again when main moved. The only conflict each time was CHANGELOG.md, where both sides add a bullet under the same ## Unreleased anchor; resolved by taking main's list and putting this PR's bullet first, as before. docs/profiles.md and zig/src/core/profile.zig both needed a merge (main added TF_GLM_PROMPT_TAIL and the #2226 wording in the same places) - resolved by keeping both sides: the registry has main's #2226 row and this PR's four rows, and the paragraph keeps main's GLM host-memory and TF_GLM_PROMPT_TAIL wording next to this PR's Gemma 4 and Spark2.5 clause.

2. The CHANGELOG line, under ## Unreleased:

Profiles: Dory's, Gemma 4's and Spark2.5's pool-page switches (TENSORFOLD_DORY_KV_PAGES,
TENSORFOLD_GEMMA4_KV_PAGES, TENSORFOLD_SPARK25_KV_PAGES) and Nemotron's paging on Metal
(TF_NEMOTRON_METAL_KV_PAGES) are registered in the profile registry; the startup line and capabilities report
them, and the values are read as before with every default unchanged (#2306, part of #1660).

3. hip is back in the first registry test's backend list, which now reads { "cuda", "metal", "hip", "mlx", "rocm", "xpu" }. I kept the other three you did not name. The test's positive assertion also changed: it used to ask e.applies(e.families[0], "cuda"), which the Metal row cannot satisfy, so it now asks the row's own scope in one unguarded loop - expectEqual(std.mem.eql(u8, backend, e.backend), e.applies(e.families[0], backend)) - which pins every row's backend instead of assuming one.

4. The docs' count. The sentence now states the count as a fact about the registry rather than as a sum of the list: "The registry names fourteen variables." The Switch enum has 14 members and entries has 15 rows, because TF_KV_PAGES has a GLM row and a Nemotron row - so "fourteen" is right for variables and the sentence claims no row count. The enumeration above it is unchanged.

5. not_zero, beside the auto and report_values notes: "Dory's row keeps the not_zero syntax: every value except exactly 0, empty text included, adds an override." That matches differs exactly - not_zero is raw != "0", and the Dory reader is set and not "0".

Full host suite on this head, on one aarch64 host (20 cores, Zig 0.17.0 from .zig-version, an empty Zig cache): zig build test --summary all - exit 0, Build Summary: 569/569 steps succeeded; 7345/7406 tests passed (61 skipped), 4m57s wall.

Also on this head:

  • zig build test-profile - exit 0 (3/3 steps, 13 tests, including the three new ones for Dory, Gemma 4 and Spark2.5, and Nemotron on Metal).
  • Lean checker, python3 tools/zig/lean_check.py, base and head: 325 problems each, and tools/zig/lean_new.py reports 0 new findings.
  • zig fmt --check on the eight changed Zig files - exit 0; git diff --check - exit 0.
  • CI on head 109fb3893: green - all seven checks (zig, zig-windows, zig-macos-dev-tools, fmt, release-paths, benchmark-clients on ubuntu-latest and windows-latest).
  • No behaviour change: each reader now resolves its switch through profile.get and applies the same rule as before (Dory: set and not "0"; Gemma 4 and Spark2.5: exactly "0" is off, which is what the old envOff did; Nemotron on Metal: exactly "1"). The registry's identical class keeps the disk-cache identity unchanged, and capabilities --json picks the rows up from the same table.

Review: GLM-5.3, two rounds. Round 1 asked for the backend-scope loop and the docs' count sentence, both applied; it also found that Dory's effect text read as if the pages were on by default, which is wrong (unset leaves every stream's caches contiguous), so the row now says "when set to anything but exactly 0". Round 2 approved with no correctness findings.

One thing to flag: the hip row you asked to keep back is a backend no registry row uses, so it only proves that a row does not apply to HIP - which is what the old assertion did.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @devopsfortunato, all of it is in: the CHANGELOG line, hip back (and the loop now pins every row's own backend), the not_zero sentence and the fourteen. It goes in the landing queue on 109fb3893 once our own cells on that head are in: main against it, with tokens equal and speed level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants