Skip to content

fix(profile): report Nemotron CUDA paging overrides - #2061

Merged
ashhart merged 2 commits into
ashhart:mainfrom
garvrao80:fix/nemotron-profile-paging
Nov 4, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
garvrao80:fix/nemotron-profile-paging

Conversation

@garvrao80

@garvrao80 garvrao80 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

A Nemotron CUDA server started with TF_KV_PAGES=0 currently prints registered overrides: none. Report that override, and exact all, for both nemotron_h and nemotron_h_omni. Keep its default-on parser: off, auto, empty text and every other spelling still select on and do not appear as overrides. GLM's same-named automatic policy retains its existing reporting.

Keep one environment lookup and disjoint family/backend policy rows. The registry still names eight variables; TF_KV_PAGES now has separate GLM and Nemotron rows. report_values explicitly describes exact-value overrides in capabilities metadata. Compile-time checks reject overlapping scopes and missing variables. The native Nemotron adapter uses the shared environment-first lookup while preserving its parser semantics, allocation and admission rules.

Review follow-up

Head 0c3a0741c, including current main at 2a7c7b7e7:

  • Extract the same native expression into private pagingOf(raw). A deterministic test compares all 16 unset/exact/fallback inputs with the original expression on every run. Four planted errors (unset default, 0, all, fallback) each fail the intended assertion. Keep the live-reader integration check too.
  • Add an Unreleased changelog entry documenting both scoped TF_KV_PAGES capability rows and optional report_values metadata.
  • Focused native tests pass on Windows and Linux. Fresh-process checks pass for 16 environment cases against both the native reader and shared lookup on each OS (64 process runs); the production capabilities serializer regression passes on Linux.
  • Lean: 425 inherited findings on current main and this head, zero new findings; formatting and diff checks pass.
  • Complete file-list audit: 145 open PRs, 55 overlaps. No new conflict paths or changed conflict boundaries. Twelve pre-existing changelog conflict blocks include the added release note; removing only that insertion restores each baseline conflict block exactly. Other inherited conflicts remain explicit in the audit; the ATC global scan is incomplete.
  • Full host/kernel-contract/golden/fuzz checks pass on both OSes: Linux 547/547 steps, 6,971/7,026 reported tests passing (55 skipped); Windows 508/508 steps, 6,436/6,573 reported tests passing (137 skipped). Plain builds pass: Linux 31/31 steps, Windows 27/27 steps. All seven GitHub CI checks pass on this head. The receipts below are for the original head, not this follow-up.

Checks on the original head

  • Head: 4c3cba3e7c84dbcb4db0b68630dc740ef647bc0c; Zig 0.17.0, Windows x86_64 and Linux x86_64 (WSL).
  • The new startup regression compiles against the original registry and fails with the intended TF_KV_PAGES=0 versus none assertion; it passes with the fix. Profile and startup tests cover both Nemotron aliases, GLM's automatic policy, exact/fallback values, backend isolation and bounded output.
  • Fresh-process validation on both Windows and Linux: 16 environment cases against both the shared lookup and the actual native Nemotron paging function, comparing with its original libc reader (64 process runs total). The capabilities regression checks the emitted JSON's environment name, both scoped policies and absent profile defaults; the built Windows CLI also emits both rows correctly. A planted overlapping family row fails the compile-time guard.
  • A clean synthetic merge with qwen CUDA: a faster opt-in paged path (TF_QWEN_KV_PAGES=256), --parallel a cap with pages set #2011 at c4eb274cd5614c2407abe09de21792fc584d9b95 passes all eight profile tests and four startup-formatting tests on Linux.
  • Linux final full check: zig build test test-kernel-contract test-golden fuzz-cortheon -j4 --test-timeout 10m --summary all passes 529/529 steps, 5,197/5,215 reported tests passing and 18 skipped (with cached work reused). The first Windows sweep passes 490/490 steps, 6,003/6,121 reported tests passing and 118 skipped; fresh Windows CI also passes on this published head. The final committed-tree Windows recheck passes 490/490 steps, with 1,033/1,044 reported tests passing and 11 skipped (cached work reused). Plain builds pass 31/31 steps on Linux and 27/27 on Windows.
  • An initial Linux sweep hit the untouched digest-cache same-size-rewrite/restored-mtime test. It did not reproduce in 30 focused runs on the pinned base, and the final full Linux sweep passes. No test was excluded or relaxed.
  • Lean: 439 inherited findings on both the pinned base and this head across 5,210 files; zero new findings. Formatting and git diff --check pass.
  • The optional server-cpu aggregate has an existing fake_serve compile error (slide_model.zig cannot import core), reproduced on the untouched base. Its CLI component builds independently; no fake-server fix is included here.
  • CI: all seven checks pass on this head, including Linux and Windows Zig, macOS developer tools, release paths, formatting and both benchmark-client jobs.

Receipt

Native paging retains its original default and exact 0/all parser. The shared lookup returns the original environment value without coercion; the Windows wrapper's only special case is unrelated HOME. This changes configuration reporting, not paging arithmetic or kernels. No live GPU/checkpoint, token-equivalence, throughput or memory qualification is claimed; no CUDA device or Mac is attached. Those are not replaced by the host checks above.

Checklist

  • One reporting/registration change with the existing policy preserved.
  • A regression that fails before the fix; original-reader comparisons and emitted capabilities checked.
  • Single-line source comments and zero new lean findings.
  • No private identities, paths, credentials or tool-attribution trailers in the publication.

@garvrao80
garvrao80 marked this pull request as ready for review November 3, 2026 19:57
@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @garvrao80, this is a tidy fix. A Nemotron server started with TF_KV_PAGES=0 now says so on its startup line instead of "none". The reader is unchanged: the same bytes from the same variable, with 0 and all as the only overrides, and the compile-time scope check keeps GLM's row and Nemotron's apart.

Two small things:

  • the new test in nemotron/cuda_native.zig runs on the test process's own environment, so it passes either way. A small pure pagingOf(raw) checked against the old expression would make it bite;
  • a CHANGELOG line would help, since capabilities --json now has two TF_KV_PAGES rows and a report_values key.

Happy to land it with those.

@garvrao80

garvrao80 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Addressed both points in a3e57fd2e and 0c3a0741c.

paging() now delegates to a private pagingOf(raw). Every test run checks 16 raw inputs against the original expression, including unset, exact 0/all, empty text, whitespace and fallback spellings. Planting an error in each of the unset, zero, all and fallback branches makes the new test fail its assertion. The live-reader integration check remains.

The Unreleased changelog entry calls out the two scoped TF_KV_PAGES rows and optional report_values metadata.

Full Linux and Windows host/kernel-contract/golden/fuzz checks pass, as do both platforms' focused/process checks. Lean adds zero findings over current main. Both plain builds pass too (Linux 31/31 steps, Windows 27/27), and all seven GitHub CI checks pass on this head.

@ashhart

ashhart commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thanks @garvrao80. pagingOf(raw) checked over 16 inputs makes the test bite, and the CHANGELOG line covers the two TF_KV_PAGES rows and report_values. Both points are in. Ready to land.

@ashhart
ashhart force-pushed the fix/nemotron-profile-paging branch from 0c3a074 to f691457 Compare November 4, 2026 17:52
@ashhart
ashhart merged commit de9f9ac into ashhart:main Nov 4, 2026
6 of 7 checks passed
@ashhart

ashhart commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thank you @garvrao80. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

@ashhart

ashhart commented Nov 5, 2026

Copy link
Copy Markdown
Owner

Thanks @garvrao80! This is in TensorFold 1.2.3, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants