Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,28 @@
### Unreleased
### 2.10.0 (Wednesday, August 26, 2026)
### Features/Bug Fixes
* Inspect hidden and nested ZIP-compatible artifacts under cumulative safety bounds.
* Report HIGH SC9 findings for executables concealed in documents or hidden/disguised artifacts.
* feat(scan): add opt-in transitive reference scanning (#225)
* feat(analyzer): add phase-1 structured skill summaries (#211)
* feat(analyzer): detect external model and provider selection as EA5 (#426)
* feat(llm): support configurable human-readable output language (#425)
* feat(report): expose the highest reported issue severity (#398)
* feat: dynamically discover analyzers and validate risk-score inputs (#74)
* fix(security): strengthen bounded inspection and fail-closed completeness handling (#393)
* fix(report): mark partial or unavailable requested LLM analysis as degraded (#291) (#362)
* fix(cli): report only the findings that drove the risk score (#375)
* fix(cli): normalize multi-skill risk scores before aggregate exit-code checks (#368)
* fix(scanner): preserve eligible findings from SKILL.md files (#381)
* fix(cli,supply-chain): parse package.json as JSON and send fatal errors to stderr (#323)
* fix(analyzer): detect whitespace variants and all os.environ read forms for E2 (1d379dc)
* fix(analyzers): reduce false positives across AE3, AE4, EA1, EA3, E5, PE3, and TM4 (#237) (#328) (#415) (#417) (#422)
* fix(pe3): require credential-store operations instead of flagging bare keyring or keychain nouns (#424)
* fix(scan): exclude inactive Git hook samples from executable analysis (#412)
* fix(nv_build): update the default served model and its token limits (#390) (#391)
* fix(lp1): tailor least-privilege remediation to the manifest type (#402)
* fix(provider): preserve the original custom CLI-provider call contract for ordinary scans
* build: move LangGraph Studio tooling to the langgraph-dev optional extra (550b9f0)
* ci: keep eligible pull-request branches current after main changes (#376)
---
### 2.9.6 (Tuesday, August 18, 2026)
### Features/Bug Fixes
Expand Down
110 changes: 110 additions & 0 deletions docs/release/skillspector-2.10.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# SkillSpector v2.10.0

Released: 2026-08-26

## Summary

SkillSpector 2.10.0 expands security coverage across concealed artifacts, referenced skills, structured skill bundles, and external model selection. It also makes incomplete analysis harder to mistake for a clean result, adds localized LLM finding text, and exposes the highest reported issue severity for downstream policy gates.

## Highlights

- Inspect hidden files and ZIP-compatible nested artifacts under cumulative safety bounds, with HIGH SC9 findings for concealed executables and provenance-preserving virtual paths.
- Add opt-in transitive reference scanning with bounded traversal, source provenance, shared budgets, and fail-closed completeness reporting.
- Recognize AISOP/AISP structured skill bundles and render report-only workflow summaries without affecting risk scores.
- Add EA5 detection for external model or provider selection, including silent coding-CLI account switches and top-level model pins.
- Add `SKILLSPECTOR_OUTPUT_LANGUAGE` for human-readable LLM finding text and `risk_assessment.max_issue_severity` for machine-readable policy gates.

## Added

- Add bounded local inspection of hidden and nested ZIP, DOCX, XLSX, and PPTX content without extracting or executing members.
- Add opt-in transitive scanning of supported skill references with `--transitive`, plus `--transitive-depth`, `--transitive-allow-prefix`, and `--transitive-deny-prefix` controls.
- Add structured skill summaries for valid AISOP/AISP bundles across terminal, Markdown, JSON, and SARIF output.
- Add dynamic analyzer discovery and validate risk-score inputs against the registered analyzer set.
- Add EA5 static findings for actionable external model or provider selection.
- Add configurable output-language instructions for discovery analyzers, the meta-analyzer, and MCP tool-poisoning analysis.
- Add `risk_assessment.max_issue_severity`, with `NONE` when no active issue is reported.

## Changed

- Move `langgraph-cli[inmem]` from the base installation to the `langgraph-dev` optional extra; the `dev` extra continues to include it.
- Update the NVIDIA Build default model to a currently served model and declare accurate limits for GLM-5.2.
- Tailor LP1 least-privilege remediation to the scanned manifest type.
- Automatically update eligible pull-request branches after changes land on `main`.

## Fixed

- Mark requested LLM analysis as degraded when any call fails or the configured provider is unavailable, flooring an otherwise `SAFE` recommendation to `CAUTION`.
- Report and baseline only the active findings that actually drove the risk score.
- Normalize serialized multi-skill risk scores before computing aggregate exit codes, with malformed values safely falling back to zero.
- Preserve eligible findings from `SKILL.md` instead of dropping them as code examples.
- Parse `package.json` as JSON for supply-chain analysis and route fatal CLI diagnostics to stderr.
- Detect whitespace-tolerant environment harvesting and all supported `os.environ` read forms.
- Reduce false positives across inactive Git hook samples, license boilerplate, wildcard tool grants, OAuth credential terminology, reference directories, and non-text artifact content.
- Require an operation tied to a keyring or keychain noun before reporting PE3 in Markdown and text prose, while preserving actionable credential-store findings.
- Preserve the original custom CLI-provider call contract for ordinary scans while forwarding explicit deadlines to providers used by bounded scan paths.

## Security

- Strengthen cumulative resource bounds, inspection-ledger completeness, finding provenance, Unicode normalization, and fail-closed behavior across scan paths.
- Keep hidden and nested artifact content local to deterministic analysis and exclude it from LLM prompts.
- Preserve deterministic security findings through filtering, suppression, recursive, transitive, MCP, and report-rendering paths.
- Surface partial provider execution and traversal truncation so incomplete deep scans cannot silently appear clean.

## Breaking Changes and Migration

- No existing CLI command, option, or report field was removed.
- LangGraph Studio users who install only the base package should install `skillspector[langgraph-dev]`; `make install-dev` and the `dev` extra continue to include this tooling.
- Custom CLI providers keep the original `complete(prompt, *, model, max_output_tokens)` contract for ordinary scans. Providers used with new deadline-bounded paths may additionally accept `timeout` as an optional keyword.

## Deprecations

- None.

## Validation

- `uv lock --check` — passed.
- `.venv/bin/pytest -q tests/unit/test_llm_utils.py tests/unit/test_create_github_release.py tests/unit/test_github_release_workflow.py tests/unit/test_wheel_contents.py` — 54 passed.
- `.venv/bin/pytest -m 'not integration and not provider' --cov=src/skillspector --cov-report=term --cov-report=xml tests/` — 2,937 passed, 13 skipped, 38 deselected, and 4 expected failures.
- `.venv/bin/ruff check src/ tests/ scripts/` — passed.
- `.venv/bin/ruff format --check src/ tests/ scripts/` — 196 files already formatted.
- Built `skillspector-2.10.0-py3-none-any.whl` and `skillspector-2.10.0.tar.gz`; `twine check` passed for both distributions.
- `skillspector --version` — reported `SkillSpector v2.10.0`.
- The GitHub release helper dry run resolved tag `v2.10.0` and the matching versioned release notes.
- `git diff --check` — passed.

## Known Limitations

- Transitive scanning remains opt-in and is limited to source types supported by the secure input handler; it is not a general-purpose web crawler and is not enabled for MCP scans.
- Nested inspection is limited to ZIP-compatible containers, enforces fixed cumulative bounds, and does not render, install, or execute nested content.
- `SKILLSPECTOR_OUTPUT_LANGUAGE` affects human-readable LLM-generated finding text only; deterministic findings and machine-readable schema values remain unchanged.
- Legacy custom CLI providers that do not accept `timeout` remain compatible with ordinary scans but cannot participate in a new path that requires an explicit provider deadline until they add that optional keyword.

## References

- [GitHub PR #74](https://github.com/NVIDIA/SkillSpector/pull/74)
- [GitHub PR #211](https://github.com/NVIDIA/SkillSpector/pull/211)
- [GitHub PR #225](https://github.com/NVIDIA/SkillSpector/pull/225)
- [GitHub PR #237](https://github.com/NVIDIA/SkillSpector/pull/237)
- [GitHub PR #291](https://github.com/NVIDIA/SkillSpector/pull/291)
- [GitHub PR #323](https://github.com/NVIDIA/SkillSpector/pull/323)
- [GitHub PR #328](https://github.com/NVIDIA/SkillSpector/pull/328)
- [GitHub commit 1d379dc](https://github.com/NVIDIA/SkillSpector/commit/1d379dca7f8e83f3785aef4954d53c56908d2ad0)
- [GitHub PR #362](https://github.com/NVIDIA/SkillSpector/pull/362)
- [GitHub PR #368](https://github.com/NVIDIA/SkillSpector/pull/368)
- [GitHub PR #375](https://github.com/NVIDIA/SkillSpector/pull/375)
- [GitHub PR #376](https://github.com/NVIDIA/SkillSpector/pull/376)
- [GitHub PR #381](https://github.com/NVIDIA/SkillSpector/pull/381)
- [GitHub PR #382](https://github.com/NVIDIA/SkillSpector/pull/382)
- [GitHub PR #390](https://github.com/NVIDIA/SkillSpector/pull/390)
- [GitHub PR #391](https://github.com/NVIDIA/SkillSpector/pull/391)
- [GitHub PR #393](https://github.com/NVIDIA/SkillSpector/pull/393)
- [GitHub PR #398](https://github.com/NVIDIA/SkillSpector/pull/398)
- [GitHub PR #402](https://github.com/NVIDIA/SkillSpector/pull/402)
- [GitHub PR #412](https://github.com/NVIDIA/SkillSpector/pull/412)
- [GitHub PR #415](https://github.com/NVIDIA/SkillSpector/pull/415)
- [GitHub PR #417](https://github.com/NVIDIA/SkillSpector/pull/417)
- [GitHub PR #422](https://github.com/NVIDIA/SkillSpector/pull/422)
- [GitHub PR #424](https://github.com/NVIDIA/SkillSpector/pull/424)
- [GitHub PR #425](https://github.com/NVIDIA/SkillSpector/pull/425)
- [GitHub PR #426](https://github.com/NVIDIA/SkillSpector/pull/426)
- [GitHub commit 550b9f0](https://github.com/NVIDIA/SkillSpector/commit/550b9f00ad1635f9b5066ac2b1c4cf399a631cfb)
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "skillspector"
version = "2.9.6"
version = "2.10.0"
description = "SkillSpector: Security scanner for AI agent skills (Claude Code, Cursor, and similar). Scans skills for vulnerabilities, malicious patterns, and security risks before installation. Supports Git repos, URLs, zips, and local directories; runs static pattern checks and optional LLM semantic analysis; outputs terminal, JSON, and Markdown reports with risk scoring."
readme = "README.md"
license = "Apache-2.0"
Expand Down
36 changes: 34 additions & 2 deletions src/skillspector/llm_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,36 @@ def _extract_json_object(raw: str) -> dict:
)


def _complete_agent_cli(
provider: object,
prompt: str,
*,
model: str,
max_output_tokens: int,
timeout: float | None,
) -> str:
"""Call a CLI provider without breaking the original extension protocol.

``AgentCLICapable`` originally exposed no ``timeout`` keyword. Preserve
that call shape for ordinary scans, while forwarding an explicit deadline
to providers used by bounded scan paths. Legacy providers therefore keep
working for existing flows and fail closed if selected for a new flow that
requires deadline support.
"""
if timeout is None:
return provider.complete( # type: ignore[attr-defined,no-any-return]
prompt,
model=model,
max_output_tokens=max_output_tokens,
)
return provider.complete( # type: ignore[attr-defined,no-any-return]
prompt,
model=model,
max_output_tokens=max_output_tokens,
timeout=timeout,
)


class _StructuredAgentCLIModel:
"""Mimics ``ChatOpenAI.with_structured_output(schema)`` for a CLI provider.

Expand Down Expand Up @@ -247,7 +277,8 @@ def _augment(self, prompt: str) -> str:

def _complete(self, prompt: str) -> str:
"""Return provider output before structured parsing begins."""
return self._provider.complete( # type: ignore[attr-defined,no-any-return]
return _complete_agent_cli(
self._provider,
self._augment(prompt),
model=self._model,
max_output_tokens=self._max_output_tokens,
Expand Down Expand Up @@ -312,7 +343,8 @@ def stream(self, *args: object, **kwargs: object) -> NoReturn:
)

def invoke(self, prompt: str) -> _AgentCLIMessage:
text = self._provider.complete( # type: ignore[attr-defined]
text = _complete_agent_cli(
self._provider,
prompt,
model=self._model,
max_output_tokens=self._max_output_tokens,
Expand Down
7 changes: 4 additions & 3 deletions src/skillspector/providers/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,11 @@ class AgentCLICapable(Protocol):
otherwise. This replaces the credential-based availability check
in :func:`skillspector.llm_utils.is_llm_available` for CLI providers.

``complete(prompt, *, model, max_output_tokens, timeout)``
``complete(prompt, *, model, max_output_tokens)``
Execute the CLI, pass the prompt via stdin, and return the
assistant's text response. Raises on any failure (fail-closed).
assistant's text response. Implementations may also accept an
optional ``timeout`` keyword to participate in bounded scans. Raises
on any failure (fail-closed).
"""

def is_available(self) -> tuple[bool, str | None]: ...
Expand All @@ -107,7 +109,6 @@ def complete(
*,
model: str,
max_output_tokens: int,
timeout: float | None = None,
) -> str: ...


Expand Down
71 changes: 70 additions & 1 deletion tests/unit/test_llm_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -419,7 +419,76 @@ def test_adapter_invoke_returns_content(self, monkeypatch: pytest.MonkeyPatch) -
) as fake_complete:
msg = get_chat_model(model="claude-sonnet-4-6").invoke("hi")
assert msg.content == "hello"
assert fake_complete.call_args[1]["timeout"] is None
assert "timeout" not in fake_complete.call_args[1]

def test_adapter_preserves_the_legacy_cli_provider_signature(self) -> None:
class _Schema(BaseModel):
verdict: str

class _LegacyCLIProvider:
DEFAULT_MODEL = "legacy-model"
SLOT_DEFAULTS: dict[str, str] = {}

def __init__(self) -> None:
self.responses = iter(["hello", '{"verdict": "safe"}'])
self.calls: list[tuple[str, str, int]] = []

def get_context_length(self, model: str) -> int | None:
return 4096

def get_max_output_tokens(self, model: str) -> int | None:
return 1024

def resolve_model(self, slot: str = "default") -> str:
return "legacy-model"

def resolve_credentials(self) -> tuple[str, str | None] | None:
return None

def is_available(self) -> tuple[bool, str | None]:
return True, None

def complete(
self,
prompt: str,
*,
model: str,
max_output_tokens: int,
) -> str:
self.calls.append((prompt, model, max_output_tokens))
return next(self.responses)

provider = _LegacyCLIProvider()
token = use_provider(provider)
try:
model = get_chat_model()
assert isinstance(model, AgentCLIChatModel)
assert model.invoke("plain").content == "hello"
structured = model.with_structured_output(_Schema).invoke("structured")
finally:
reset_provider(token)

assert structured == _Schema(verdict="safe")
assert [call[1:] for call in provider.calls] == [
("legacy-model", 1024),
("legacy-model", 1024),
]

def test_explicit_timeout_requires_cli_provider_timeout_support(self) -> None:
class _LegacyCLIProvider:
def complete(
self,
prompt: str,
*,
model: str,
max_output_tokens: int,
) -> str:
return "unreachable"

model = AgentCLIChatModel(_LegacyCLIProvider(), "legacy-model", 1024, timeout=1.0)

with pytest.raises(TypeError, match="timeout"):
model.invoke("bounded")

def test_structured_output_parses_and_validates(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("SKILLSPECTOR_PROVIDER", "claude_cli")
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading