Skip to content

test(hip): re-derive every gate-5 record with ROCm 10.1 - #1979

Merged
ashhart merged 2 commits into
ashhart:mainfrom
laudney:pr-hip-records-rocm101
Nov 3, 2026
Merged

ashhart merged 2 commits into
ashhart:mainfrom
laudney:pr-hip-records-rocm101

Conversation

@laudney

@laudney laudney commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Part of #1977. Independent of #1978 (its own records are already ROCm 10.1 counts).

Every HIP gate-5 record re-derived with ROCm 10.1's hipcc, so the limit gate checks something again on a ROCm 10.1 host.

before (main) after (this PR)
hip-kernel-contract-limits with ROCm 10.1 (gfx1030, gfx1100, gfx1151, gfx1201) FAIL: 502 problems, every record "stale" PASS: every record current, every control red
hip-kernel-contract on an AMD Radeon AI PRO R9700 (gfx1201), its live limit gate the stale records count as limit problems 52 kernels: 52 pass gates 1-4, 0 fail, 2 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems
Records' compiler ROCm 7.13's hipcc ROCm 10.1's hipcc (HIP 7.16), named in the contract guide
Kernels, launches, gates 1 to 4 unchanged

What this changes

The 148 HIP launch records in zig/src/core/contract/hip_records*.zig held what ROCm 7.13's hipcc compiled. A host with
ROCm 10.1 compiles different register counts for most launches, so gate 5 reported every record stale and the
limit check could not tell a real overrun from a compiler difference. This PR takes ROCm 10.1 as the reference and
rewrites each record's four .usage rows from the audit's own RECORD lines. No value is typed by hand: a script
replaced the rows in the order check() walks the records and refused any record whose source name differed from
the audit's. 127 records changed; 21 already matched. No kernel source, launch, bound or control changes.

docs/kernel-contract.md now names the reference: ROCm 10.1's hipcc, which ROCm 10.1 installs under core/bin of
its root (pass that path to -Dhipcc; the bin/hipcc wrapper there loses the device math headers).

Receipt

  • Environment: TensorFold upstream/main d8aeab9 + this commit; ROCm 10.1 (HIP 7.16); AMD Radeon AI PRO R9700
    (gfx1201); Zig 0.17.0.
  • zig build hip-kernel-contract-limits -Dhip=true -Dhipcc=<rocm>/core/bin/hipcc: before FAIL kernel-contract-limits: 502 problems; after PASS kernel-contract-limits: every registered launch fits gfx1030, gfx1100, gfx1151 and gfx1201, every record current, every control red.
  • zig build hip-kernel-contract -Dhip-arch=gfx1201 on the R9700: SUMMARY gfx1201 52 kernels: 52 pass gates 1-4, 0 fail, 2 known findings; 5 controls: 5 red as planted, 0 missed; 0 limit problems (exit 0); the 2 known findings are the existing prompt-tile non-finite entries.
  • zig build test-kernel-contract: 164/164 passed.
  • Not run: GPU gates on gfx1030, gfx1100 and gfx1151 (no such GPU here; their records come from the compile-only
    audit, which covers all four chips).
  • Exactness, decode and prompt speed: n/a, no kernel or launch changes.

Merging with other HIP PRs

Open PRs that add or change HIP records (for example #1612, #1614, #1622, #1658, #1694, #1877) touch the same record
lists. Whichever lands second takes a plain merge of the neighbouring lines; a record a PR adds with ROCm 7.13 counts
then reads stale under this reference until hip-kernel-contract-limits re-derives it (its RECORD line is the value).

@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @laudney. Decision: ROCm 10.1's hipcc becomes the single reference for HIP gate-5 launch records. The change is record values only (127 of 148 launches: VGPR/SGPR counts and 3 scratch rows; LDS, bounds and waves unchanged; names, launches and controls byte-identical), with no kernel, build or test file. Contract runs on an older ROCm will now fail loudly as stale rather than pass on old numbers, and serving is untouched. It's in the landing queue.

laudney and others added 2 commits November 3, 2026 06:05
The HIP launch records held ROCm 7.13's compiled usage, so on a host
with ROCm 10.1 every record read stale (502 limit problems) and gate
5 checked nothing there; ROCm 10.1 is now the reference, and each of
the 148 records' usage is rewritten from the limits audit's own
RECORD lines over 10.1's code objects for gfx1030, gfx1100, gfx1151
and gfx1201, with the reference hipcc named in the contract guide.
@ashhart
ashhart force-pushed the pr-hip-records-rocm101 branch from 97075bf to 1767294 Compare November 3, 2026 06:05
@ashhart
ashhart merged commit cb972b8 into ashhart:main Nov 3, 2026
0 of 7 checks passed
@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thank you @laudney. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

laudney added a commit to laudney/TensorFold that referenced this pull request Nov 3, 2026
The ROCm 10.1 gate-5 records (ashhart#1979) and the MoE gate and router
contracts reached main, and they touch the same shared lists as this
branch, so main is merged in, not rebased, to keep the open PR's
history. The contract runner keeps both sets of kernels, the HIP
record list ends with the router records and then the dense GEMM
records, and the CHANGELOG keeps every Unreleased entry.
@gilby

gilby commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Since #1979 makes ROCm 10.1 the reference compiler, here is how the engine runs under it against Ubuntu 26.04's own ROCm 7.1 packages (hipcc 7.1.1, clang 21; ROCm 10.1 is HIP 7.16, clang 24).

Setup: the same source (the #1658 branch rebased on main cfcbecb), built with each toolchain, on one Strix Halo iGPU (gfx1151), zig 0.17.0, probe Flash Next checkpoint, --prompt-cache-gib 0, page cache dropped before every server start, a fresh server per arm, arms alternated A, B, C, A, B, one model process on the box. Clients: bench_openai.py (256 tokens, 3 reps, T=0), bench_concurrent.py --alone --serial, bench_cache.py --require-cold. Arm A is ROCm 7.1 hipcc with the 7.1 runtime. Arm B is ROCm 10.1 hipcc with the 10.1 runtime (HIP 7.16, ROCr 1.21, checked in the process's mappings). Arm C is the 10.1-built binary on the 7.1 runtime.

tok/s (T=0) A: 7.1 B: 10.1 C: 10.1 binary, 7.1 runtime B vs A
decode, code, 1 stream 52.77 (52.75, 52.79) 52.02 (51.95, 52.08) 52.10 -1.4%
decode, chat, 1 stream 46.55 (46.52, 46.58) 45.85 (45.79, 45.91) 45.93 -1.5%
code, 4 streams 93.35 (93.3, 93.4) 91.05 (91.0, 91.1) 91.2 -2.5%
chat, 4 streams 76.35 (76.3, 76.4) 74.50 (74.4, 74.6) 74.5 -2.4%
cold 32K prompt, first token 84.2 s (84.0, 84.3) 84.6 s (84.5, 84.7) 84.9 s +0.5%
cold 8K prompt, first token 23.5 s (23.2, 23.8) 23.7 s (one run), 72.2 s (the other) 24.1 s +1% (see below)

Means with the two runs in brackets. One B run of the cold 8K prompt took 72.2 s against 23.7 s in the other; I have no cause for it and did not repeat it, so I leave it out of the comparison. Every token_sha256 is identical across all five arms on every cell, so the compiler changes no token.

Reading: ROCm 10.1 is 1.4 to 2.5% slower than Ubuntu's 7.1 here, and the 10.1 binary on the 7.1 runtime reads the same as on the 10.1 runtime, so the gap comes from the compiler and not the HIP runtime. Prompt speed is within a percent. Not run: any other GPU or family.

@ashhart

ashhart commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks @laudney! This is in TensorFold 1.2.2, out now: https://github.com/ashhart/TensorFold/releases/tag/v1.2.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants