fix(rules): CSKILL pattern-blindness and OAI-201 tracing detection - #214
Merged
sairenchristianbuerano merged 1 commit intoSep 21, 2026
Merged
Conversation
ivanpaghubasan
requested review from
ericksondelacruz,
jhumel-code and
sairenchristianbuerano
September 18, 2026 12:47
ivanpaghubasan
marked this pull request as draft
September 18, 2026 12:48
sairenchristianbuerano
approved these changes
Sep 21, 2026
ivanpaghubasan
marked this pull request as ready for review
September 21, 2026 03:19
Two confirmed false-positive fixes from outreach feedback (~2 months open). Bug 1 — CSKILL-050, CSKILL-082, and CSKILL-060 (via shared predicate) matched on tool name only, never the grant Pattern, so a narrowly- scoped grant like Bash(git status:*) fired identically to unrestricted Bash(*). Added skillGrantIsUnrestricted() encoding Claude Code's real per-tool permission semantics (verified against docs): Bash/PowerShell and WebFetch have genuine pattern restrictions, Edit has genuine gitignore-pattern restrictions, but Write/NotebookEdit path rules are accepted and never consulted, so those two remain always-unrestricted. New schema key skill_allows_unrestricted_tool added rather than redefining skill_allows_tool in place, per CLAUDE.md's predicate- renaming discipline. Old key retained, documented as pattern-blind. Schema version bumped 16 -> 17. Bug 2 — OAI-201 only recognized add_trace_processor/set_trace_processors and an OPENAI_AGENTS_DISABLE_TRACING string literal, missing two documented disable mechanisms: set_tracing_disabled(True) and RunConfig(tracing_disabled=True). Confirmed a live false positive in our own corpus (testdata/corpus/basic-openai-agent calls set_tracing_disabled(True) and OAI-201 fired on it, untested). Added a new source-level fire/silent scanner test for OAI-201, closing known gap (d) in testdata/rules-fixture/CLAUDE.md for this rule specifically. Fixed unrelated pre-existing merge-conflict markers in schema.yaml (lines 357/372/380, from PYD-104) found while editing the file. Verified: go build/test clean, check-rules-sync.sh shows no drift on any touched file (pre-existing drift elsewhere is unrelated/untouched).
ivanpaghubasan
force-pushed
the
fix/cskill-050-pattern-and-oai-tracing
branch
from
September 21, 2026 03:22
dfae860 to
dc69ee9
Compare
Collaborator
Author
|
Note: the sync-check failure here is pre-existing, not introduced by this PR — the fixture-only and content-drift files all trace back to PR #218 (recover/rule-fixtures, external contribution), which added significant new rule coverage but left the fixture out of sync with production in many places. None of this PR's changes appear in the failure list. Flagging as a separate cleanup effort. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two confirmed false-positive fixes from outreach feedback (~2 months open).
Bug 1 — CSKILL-050, CSKILL-082, and CSKILL-060 (via shared predicate) matched on tool name only, never the grant Pattern, so a narrowly- scoped grant like Bash(git status:) fired identically to unrestricted Bash(). Added skillGrantIsUnrestricted() encoding Claude Code's real per-tool permission semantics (verified against docs): Bash/PowerShell and WebFetch have genuine pattern restrictions, Edit has genuine gitignore-pattern restrictions, but Write/NotebookEdit path rules are accepted and never consulted, so those two remain always-unrestricted.
New schema key skill_allows_unrestricted_tool added rather than redefining skill_allows_tool in place, per CLAUDE.md's predicate- renaming discipline. Old key retained, documented as pattern-blind. Schema version bumped 16 -> 17.
Bug 2 — OAI-201 only recognized add_trace_processor/set_trace_processors and an OPENAI_AGENTS_DISABLE_TRACING string literal, missing two documented disable mechanisms: set_tracing_disabled(True) and RunConfig(tracing_disabled=True). Confirmed a live false positive in our own corpus (testdata/corpus/basic-openai-agent calls set_tracing_disabled(True) and OAI-201 fired on it, untested).
Added a new source-level fire/silent scanner test for OAI-201, closing known gap (d) in testdata/rules-fixture/CLAUDE.md for this rule specifically.
Fixed unrelated pre-existing merge-conflict markers in schema.yaml (lines 357/372/380, from PYD-104) found while editing the file.
Verified: go build/test clean, check-rules-sync.sh shows no drift on any touched file (pre-existing drift elsewhere is unrelated/untouched).