Skip to content

Agent tools: no upper caps, instrument flags pass through, bench seed, web tools - #22

Merged
AdvayMonga merged 1 commit into
mainfrom
feat/tool-limits
Oct 10, 2026
Merged

AdvayMonga merged 1 commit into
mainfrom
feat/tool-limits

Conversation

@AdvayMonga

Copy link
Copy Markdown
Owner

The user's rule: every tool works as intended, the agent controls it however it wants, and no search space is cut. Cost is the budget's job, not an argument cap.

Changes

  • trace / kernel / hostprof:
    • Integers keep only a lower bound (requests, prompt length, output tokens, launch count/skip, seconds, rate).
    • kernel: any regex, or none (profiles every kernel), and any ncu section set.
    • New nsys_args (replaces the default nsys flags), ncu_args and pyspy_args: the instruments' own flags, passed as argv items.
    • Safety: the command line is never a shell and it runs inside the jail, where the agent's own code already runs. So the old character-set rule on the regex protected nothing.
  • bench: takes seed (seen split only), recorded in its ledger config. submit and the held-out guard are unchanged.
  • Agent built-ins: WebSearch and WebFetch are enabled for Claude.

Not here

  • Open internet for the agent's shell. srt refuses an allow-all domain list (* is rejected), so on macOS the agent's shell still reaches only the API host. Open internet comes with the workbench container (lab: container jail on Linux; runtime design (RUNTIME.md) #19 and the stack after it).
  • Under srt, WebFetch is limited to the same allowlist.
  • Corpus workloads can't be larger than the corpus class (sampling is without replacement); synthetic has no limit.

Tests: lab suite passes; the engine-side test_lab_profile.py passes (11); ruff is clean.

🤖 Generated with Claude Code

…; web tools

- trace/kernel/hostprof: integers keep a lower bound only; any regex and ncu set; nsys_args, ncu_args and
  pyspy_args pass the instruments' own flags as argv items (never a shell, run in the jail).
- bench takes `seed` (seen split), recorded in its config.
- Claude's WebSearch and WebFetch are enabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AdvayMonga
AdvayMonga merged commit d6d1dac into main Oct 10, 2026
3 checks passed
@AdvayMonga
AdvayMonga deleted the feat/tool-limits branch October 10, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant