Skip to content

fix(analyzers): reduce false positives for negated safety constraints#254

Open
koriyoshi2041 wants to merge 1 commit into
NVIDIA:mainfrom
koriyoshi2041:fix-negated-safety-false-positives
Open

fix(analyzers): reduce false positives for negated safety constraints#254
koriyoshi2041 wants to merge 1 commit into
NVIDIA:mainfrom
koriyoshi2041:fix-negated-safety-false-positives

Conversation

@koriyoshi2041

Copy link
Copy Markdown

Refs #251

Problem

Static scans can currently turn benign safety-policy prose into high-risk findings. In a local fixture, text such as must not access credentials and Do not modify this skill's own files, plus scoped cleanup of this tool's own cache path, produced PE3, RA1, and TM1 findings with a HIGH / DO_NOT_INSTALL recommendation.

Fix

This adds regression coverage for that false-positive class and narrows the static filters so:

  • PE3 and RA1 matches are skipped when the matched phrase is explicitly negated in nearby policy prose.
  • scoped user-cache cleanup under ${HOME}/.cache/... or ~/.cache/... is treated like the existing low-risk cache cleanup path instead of destructive arbitrary deletion.

Test

uv run --extra dev pytest -q tests/nodes/analyzers/test_static_false_positive_controls.py tests/nodes/analyzers/test_binary_and_pe3_filtering.py tests/unit/test_patterns_new.py::TestToolMisuse tests/unit/test_patterns_new.py::TestRogueAgent
# 97 passed

uv run --extra dev ruff check src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py tests/nodes/analyzers/test_static_false_positive_controls.py
# All checks passed!

uv run --extra dev ruff format --check src/skillspector/nodes/analyzers/static_patterns_privilege_escalation.py src/skillspector/nodes/analyzers/static_patterns_rogue_agent.py src/skillspector/nodes/analyzers/static_patterns_tool_misuse.py tests/nodes/analyzers/test_static_false_positive_controls.py
# 4 files already formatted

I also re-scanned the local benign fixture after the patch: score=0, severity=LOW, recommendation=SAFE, finding_count=0.

Risk

The negation guard is intentionally narrow: PE3 and RA1 only. The cache-cleanup handling is scoped to user cache paths, not arbitrary recursive deletion. This addresses a concrete subset of #251 rather than every false-positive case in that issue.

Signed-off-by: kigland <shuaizhicheng336@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant