GPU skip guards: run a kernel, and make RIFT_CI_REQUIRE_GPU fatal in the last two blocks - #371
Merged
Merged
Conversation
oshaughnessy-junior
deployed
to
private-review-dispatch-rift
September 19, 2026 12:47 — with 
GitHub Actions
Active
GitHub Actions
Active
Found by sweeping for the defect class rather than for the spelling. PR #370 swept for `importorskip('cupy')` and fixed the two sites that matched. This is a third hand-rolled guard doing the same job, which that sweep could not see. It is NOT the importorskip defect -- it imports cupy in a try and already fails under RIFT_CI_REQUIRE_GPU=1. Its check is `getDeviceCount() >= 1`, which asks whether a device exists, not whether this cupy can build a kernel for it. Measured on ldas-pcdev2 2026-09-19, cupy 12.0.0, where CUDA slot 1 is a Blackwell cc 12.0: CUDA_VISIBLE_DEVICES before after 0 (A100) 1 passed 1 passed 1 (Blackwell) 1 failed, CompileException 1 skipped 1 + RIFT_CI_REQUIRE_GPU=1 1 failed 1,0 (bad first slot) 1 failed, CompileException 1 passed 1,2 (bad first slot) 1 failed, CompileException 1 passed The last two rows are the second thing the old guard could not do: it never chose a slot, so it ran on the default one and died with a usable card in the list. `require_rift_backend=True` is deliberately NOT passed, and the docstring now gives the measurement rather than an inference. An earlier draft argued from the exception TYPE -- CompileException rather than the KeyError that signals RIFT's import-time fallback -- which proves nothing, because the CompileException fires first and would mask a KeyError either way. What settles it: RIFT does fall back to numpy at "1,0" and "1,2" (xpy_default is numpy, cupy_here False, both measured), and this test passes regardless, because it takes xpy as an argument and never reaches SphericalHarmonics_gpu._coeffs. On ldas-grid (cupy present, no driver) the leg skips, and fails under RIFT_CI_REQUIRE_GPU=1, on pytest 9.1.1 and 8.3.5 alike. Collection floor unchanged at 171. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… and one list decides both the count and the listing Four blocks score skips by REASON. Two carried an "under RIFT_CI_REQUIRE_GPU=1 any skip is bad" branch and two did not: time-marginalization and peak-local. All four do now. THIS CLOSES NOTHING THAT WAS OPEN. A collection-time marker census over all 292 items in the four covered files finds zero skip/skipif/xfail markers, no fixtures and no direct pytest.skip: every skip they can emit is either gpu_slot_probe.no_gpu or an importorskip of a non-cupy module. The point is to keep that true without depending on it, since the protection lives entirely in test-file Python. What the reason-match alone accepts, measured by decorating an existing peak-local test with `skipif(True, reason="no cupy on this host")` -- a skip that does not honour the flag -- and scoring the real run. ON A GPU HOST, which the first version of this message failed to say: ldas-pcdev2, CUDA slot 0. RIFT_CI_REQUIRE_GPU=1 BAD=1 gate REJECTS (base: BAD=0, ACCEPTED) flag unset BAD=0 gate ACCEPTS (unchanged, as it must be) On ldas-grid the same mutation cannot show this: with the flag set the GPU legs themselves fail, so the gate exits at the pytest-rc check and BAD is never computed, on this commit and on the base alike. Collection stayed at the pinned 121 in every arm, so the count guard does not catch this shape. THE COUNT AND THE LISTING WERE TWO SEPARATE GREPS, in all four blocks, and on the no-flag path they disagreed: the count was reason-filtered and the listing was not, so the gate said "1 unacceptable" and printed two lines, the first of them an acceptable skip. That is the no-flag path every GitHub PR run takes, and one acceptable skip is always present there. One list now decides both. The messages also claimed `pytest -rs` groups equal REASONS. It groups by (file:line, reason): two skips with the same reason at different lines print two lines, and one parametrized decorator prints one line reading [3]. Measured identically on pytest 9.1.1 and 8.3.5. Corrected in all four. NOT CLOSED, and not claimable as closed: `xfail(run=False)` disables a test with no SKIPPED line and no change in the collection count, and is accepted with the flag set. What catches it is a passed-floor, which .travis/test-q-window-stencil.sh and .travis/test-core-units.sh both have and this script has nowhere. An earlier draft of this message said "only the skip guard can" catch this shape; that was an overclaim. .travis/test-q-window-stencil.sh is left out. It runs only from ci.yml's q-window-stencil-check on a GitHub runner, which has no cupy and never sets the flag. It also scores by junit counts rather than reasons, so a new skip there already fails on EXPECTED_PASSED=75. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
no_gpu's docstring told the next reader where the RIFT_CI_REQUIRE_GPU rule is
enforced. One day later three of its statements were false, two of them because
of this branch:
* "only for the zero-likelihood stand-in and e2e gates" -- now all four blocks.
* "for these two files" -- the band-limited leg makes three callers.
* ".travis/test-q-window-stencil.sh ... score skips by REASON alone" -- it was
never true. That script has no reason matching at all
(`grep -E "grep.*(cupy|gpu|cuda)"` on it returns nothing); it scores by junit
counts, EXPECTED_PASSED=75 and MAX_SKIPS=3.
Rewritten with what is true today and a date, and the reason to keep the
in-process fail is now the one that does not expire: the shell cannot see a
`pytest <file>` run by hand on the GPU node, which is what someone does to
reproduce a CI failure, and that run would report green with the device lane
skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
oshaughnessy-junior
force-pushed
the
claude/gpu-skip-guards-require-gpu
branch
from
September 19, 2026 13:58
e32dab0 to
19c3d5a
Compare
oshaughnessy-junior
had a problem deploying
to
private-review-dispatch-rift
September 19, 2026 13:58 — with
GitHub Actions
Error
GitHub Actions
Error
oshaughnessy-junior
marked this pull request as ready for review
September 19, 2026 13:58
oshaughnessy-junior
deployed
to
private-review-dispatch-rift
September 19, 2026 13:58 — with
GitHub Actions
Active
GitHub Actions
Active
![interhost-cross-review-canary-v2[bot]](https://avatars.githubusercontent.com/u/274802661?s=60&v=4){kind=small}
oshaughn
deployed
to
private-review-dispatch-rift
September 19, 2026 16:38 — with
GitHub Actions
Active
GitHub Actions
Active
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #370, which left one item open. Three commits, each droppable.
1. A third guard with the same job
#370 swept for
importorskip('cupy').test_time_marginalization_quadrature.py::_cupy_or_skipis hand-rolled, so that sweep missed it. It already fails under
RIFT_CI_REQUIRE_GPU=1, so itis not the #370 defect, but it checks
getDeviceCount() >= 1: whether a device exists, notwhether this cupy can build a kernel for it.
ldas-pcdev2, cupy 12.0.0, CUDA slot 1 a Blackwell cc 12.0:
CUDA_VISIBLE_DEVICESCompileExceptionRIFT_CI_REQUIRE_GPU=1CompileExceptionThe old guard never chose a slot, so it ran on the default one with a usable card in the list.
require_rift_backend=Trueis not passed: RIFT does fall back to numpy at1,0and1,2(both flags measured), and this test passes regardless, because it takes
xpyas an argumentand never reaches
SphericalHarmonics_gpu._coeffs.2. The branch, and a bug the first draft of it introduced
Four blocks in
.travis/test-integrate.shscore skips by reason. Two carried an "underRIFT_CI_REQUIRE_GPU=1any skip is bad" branch; two did not. All four do now.This closes nothing that was open. A collection-time marker census over all 292 items in the
four covered files finds zero skip markers, no fixtures and no direct
pytest.skip. The branchkeeps that true without depending on it.
What the reason-match alone accepts, measured on ldas-pcdev2, CUDA slot 0, by decorating an
existing peak-local test with
skipif(True, reason="no cupy on this host"):RIFT_CI_REQUIRE_GPU=1Collection stayed at 121, so the count guard does not catch this shape.
Two message defects, both now fixed in all four blocks. The count and the listing were separate
greps, so on the no-flag path the count was reason-filtered and the listing was not: the gate
said "1 unacceptable" and printed two lines, the first an acceptable skip. That is the path every
GitHub PR run takes. One list now decides both. The messages also said
pytest -rsgroups equalreasons; it groups by (file:line, reason), measured on 9.1.1 and 8.3.5.
Not closed:
xfail(run=False)disables a test with noSKIPPEDline and no change incollection count, and is accepted with the flag set. A passed-floor catches it;
test-q-window-stencil.shandtest-core-units.shhave one and this script has none anywhere..travis/test-q-window-stencil.shis left out: it runs only on a GitHub runner with no cupy andno flag, and scores by junit counts rather than reasons, so a new skip there already fails on
EXPECTED_PASSED=75.3. The docstring commit 2 falsified
no_gputold the reader where the rule is enforced. One day later three of its statements werefalse, two because of commit 2 here. Rewritten with a date.
Gates
ldas-grid, rebased onto
d6ba261a6:test-q-window-stencil.shPASS (78/75/3),test-ci-roster.pyPASS,_TMARG171 collected and 170 passed / 1 skipped / ACCEPTED,peak-local 121 collected and 120 passed / 1 skipped / ACCEPTED.
Reviewed adversarially; five findings, all fixed above.
🤖 Generated with Claude Code