Skip to content

Add Trustabl Agent Scanner to CI - #59

Closed
trustabl-kathrina wants to merge 1 commit into
UditAkhourii:mainfrom
trustabl-kathrina:add-trustabl-action
Closed

trustabl-kathrina wants to merge 1 commit into
UditAkhourii:mainfrom
trustabl-kathrina:add-trustabl-action

Conversation

@trustabl-kathrina

Copy link
Copy Markdown

We came across your repo and we like how you're building a framework for agents to perform more creative and interdisciplinary work through techniques like tree-of-thought and pruning. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.

  1. [HIGH] Skill claims cryptographic operations
    File: skills/adhd/SKILL.md
    What it means: This skill's name or description claims a cryptographic operation — encrypting, decrypting, signing, hashing, or certificate handling — but states no concrete primitive, key source, or algorithm bound.

Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.

Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)

@UditAkhourii

Copy link
Copy Markdown
Owner

Thanks for taking a look at the repo — genuinely appreciate the effort that went into the workflow itself (SHA-pinned action, persist-credentials: false, scoped permissions, checksum-verified binary download; that's more care than most drive-by CI PRs show).

That said, we're going to pass.

The finding motivating the PR is a false positive. CSKILL-080 substring-matches sign against the skill description:

- skill_description_has_text: [crypto, encrypt, decrypt, sign, hash, certificate, ...]

Our skills/adhd/SKILL.md description contains "design" ("open-ended design, architecture, naming..."). There is no cryptography anywhere in that skill. Word-boundary matching would eliminate that whole class of hit — worth fixing, since it fires at severity: high.

Beyond that, for this repo specifically:

  • The scan job takes pull-requests: write + security-events: write on every push and PR, which is standing write access for third-party code.
  • With continue-on-error: true and severity-threshold: none it can't gate anything, so it's output-only.
  • version: v0.1.6 pins the CLI two releases behind current (v0.1.9) while the action is v0.4.1; we have no Dependabot config, so both pins would rot.
  • We already run CodeQL and dependency-review.

Happy to revisit once the skill rules are tighter. Thanks again, and good luck with the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants