[CI] Bazel: Phase 0.5 — tag codegen tool (demo, stage 1 of 3) - #2
Draft
Kangyan-Zhou wants to merge 2 commits into
Draft
[CI] Bazel: Phase 0.5 — tag codegen tool (demo, stage 1 of 3)#2Kangyan-Zhou wants to merge 2 commits into
Kangyan-Zhou wants to merge 2 commits into
Conversation
Phase-0.5 of the bazel migration. Builds the bridge between sglang's
existing run_suite.py CI registry (register_cuda_ci(suite=..., est_time=...)
decorations on test_*.py files) and Bazel's tag-based test selection.
Components:
- scripts/ci/generate_bazel_tags.py: AST-parses test_*.py files for
register_*_ci(...) calls at module scope. Emits a Starlark dict mapping
repo-relative test path to {suite, est_time, nightly}. Supports a
--check mode for pre-commit / CI gating against staleness.
- tools/sglang_test_registry.bzl: generated registry, 629 entries
spanning test/registered/ + test/srt/ + test/manual/.
- tools/pytest.bzl: pytest_test now injects "sgl-suite-<suite>" and
"est_time:<n>" tags onto each generated py_test from the registry.
User-supplied tags compose additively (no overwriting). Tests without
a register_*_ci decoration get no extra tags.
Validated on H100:
- 629 entries generated from existing decorations.
- bazel query "attr(tags, sgl-suite-stage-b-test-1-gpu-small, //test/...)"
returns the 9 currently-wired tests across 3 packages.
- bazel query --output=build //test/registered/quant:test_int8_kernel shows
tags = ["1-gpu", "est_time:15", "gpu", "sgl-suite-stage-b-test-1-gpu-small"]
(user tags + registry-injected tags coexist).
- 3 prototype 1-GPU tests still pass.
- --check correctly detects staleness (tampered file -> exit 1).
Limitations (documented in script):
- Cuda > cpu > amd > npu priority when multiple registrations exist;
AMD-specific suites lost in the demo registry.
- Non-literal kwargs are skipped with a stderr warning.
- disabled=True registrations are excluded.
Next layer (feat/bazel-pr-test-yml) wires --test_tag_filters into
pr-test.yml for stage-a-test-cpu.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses critical findings from the silent-failure-hunter and code-reviewer agents on commit 36fe6e7. Five fixes, all gated against the same failure mode (a test that disappears from the Bazel registry without anyone seeing it): - Positional argument support (was the root cause of 13 false positives in the first pass). _PARAM_ORDER mirrors python/sglang/test/ci/ci_register.py so positional calls like `register_cpu_ci(1.0, "stage-a-test-cpu")` bind correctly. Registry now has 642 entries (up from 629). - Hard-fail (exit 1) on every silent-skip path: parse errors, UnicodeDecodeError, non-literal kwargs, **kwargs splats, duplicate args, wrong-type suite/est_time. Each error logged with file:line. - Non-top-level register_*_ci calls (inside if/try/def blocks) are now fatal errors with a clear message, surfaced via ast.walk pre-pass. - json.dumps() for every string value in the generated Starlark output. Defense in depth against quote/backslash/newline characters. - est_time accepts both int and float (sglang's CPU tests use 1.0). Rejects bool. Rounds float to int for the registry. Pre-commit hook wiring (.pre-commit-config.yaml): - New `check-bazel-test-registry` hook runs `--check` on test/.../*.py, the script itself, or the registry file. Without it, --check was load-bearing UI with no enforcement. Validated on H100: all 3 prototype 1-GPU tests still pass; --check is byte-stable on a fresh regen. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Demo stack — stage 1 of 3 validating the Bazel test-runner migration plan.
This layer bridges register_cuda_ci/cpu_ci/amd_ci decorators to Bazel test tags via an AST-parsing codegen tool. Generates a 642-entry Starlark registry; pytest_test macro injects sgl-suite-* and est_time:* tags onto each py_test.
Components:
Validated on H100 (lmsysorg/sglang:dev container kangyan_bazel_test):
Reviewed by code-reviewer + silent-failure-hunter agents (findings in chat). Five fixes landed in df4b52f: positional arg support, hard-fail on every silent-skip path, ast.walk for non-top-level registrations, json.dumps for Starlark output, --check wired into pre-commit.
Stack:
🤖 Generated with Claude Code