Skip to content

Latest commit

 

History

History
418 lines (345 loc) · 21.9 KB

File metadata and controls

418 lines (345 loc) · 21.9 KB

GuardStack

Five safety gates for LLM agents, wired as one lifecycle — with the numbers to prove each one earns its place.

from guardstack import GuardStack

gs = GuardStack.from_file("guardstack.yaml")

v = gs.check_input(user_text)                 # 1  prompt injection
if v.blocked:
    return v.message

d = gs.check_tool("issue_refund", args)       # 2  grant   3  approval
if d.needs_approval:
    return hand_to_a_human(d.reason)
if not d.allowed:
    return "I can't do that."

r = gs.execute(lambda: payments.refund(**args),   # 4  retry / circuit
               once=("issue_refund", args))      # 3c did this already happen?
return gs.check_output(model_reply).text          # 5  secret + PII redaction

once= is gate 3c, new in 1.3.0: the same refund arriving as a second ticket, a second request or a second worker is refused before it reaches your provider. It is not key= — that is the circuit-breaker key. The two are spelled differently on purpose: once= names the side effect, key= names the dependency a breaker counts failures against. They are easy to read as the same thing, so they do not share a name.

Every gate writes to one hash-chained audit log, so "what did the agent try, and under which rule was it allowed" is answerable from one file.


What it stops

Run PYTHONPATH=. python examples/support_agent.py — offline, no API key. Trimmed to the scenes that carry the argument; every line below is copied from a real run.

── Large refund — a human decides
  2+3 tool gate          HELD FOR HUMAN — refund of 18900c is above the 5000c auto limit
     approval token      <single-use, 43 chars, different every run>
  → user sees            A colleague is reviewing this refund.

── Prompt injection in the customer message
  1 input guard          BLOCKED: ignore_previous, exfil_system_prompt
  → user sees            Request blocked by input guard (ignore_previous, exfil_system_prompt).

── The model tries to hand back a live key
  5 output guard         REDACTED ['openai_key']
  → user sees            Delivered. (debug: api key [REDACTED_OPENAI_KEY])

── The human approves the held refund
  approve() returns      issue_refund amount_cents=18900
  caller's dict says     issue_refund amount_cents=900000000
  execute                a.args — the values the human saw. The caller's dict is not evidence of anything.
  approval               allowed=True — approved by lead@example.com: goodwill, customer since 2019 (held at t=2)
  same token, again      allowed=False — no held call matches this token: it was never issued, has already been used, or belongs to another GuardStack

── An agent stuck in a loop stops itself
  call 1                 allowed
  call 2                 allowed
  call 3                 STOPPED — lookup_order called 3 times with identical arguments in task 'ticket-4471': that is a loop, not progress (loop_threshold=3)

── audit
  chain                  True — the chain is intact
  entries                10
    #2 issue_refund   deny   refund of 18900c is above the 5000c auto limit
    #4 issue_refund   allow  approved by lead@example.com: goodwill, customer since 2019 (held at t=2)
    #5 <unknown>      deny   no held call matches this token: it was never issued, has already been used, or belongs to another GuardStack
    #8 lookup_order   deny   lookup_order called 3 times with identical arguments in task 'ticket-4471': that is a loop, not progress (loop_threshold=3)

Why the token is described rather than printed. It is secrets.token_urlsafe(32), so it is a different 43-character string on every run — pasting one here would be a value no reader could ever reproduce. That it changes is the property you want: through 1.2.0 the token was a digest of the tool, the arguments, the step and the config fingerprint, every ingredient of which is public, so anyone who could see a held call could compute its token offline. A token you can compute is an identifier, not an authorisation, and this one travels through Slack.

The approval scene is the point. A human approves the call they were shown, and approve() hands back the tool and arguments that were held — so the thing you execute comes from the hold, not from a dict the caller has been holding across the minutes a human spent deciding. Above, that dict was changed to 900,000,000c mid-decision; approve() still returns 18,900c, and 18,900c is what entry #4 records. Before 1.2.0 the caller's dict was the only thing an integrator could run, and the audit log recorded the amount the human saw — not wrong-by-omission, but wrong evidence. The replay of a spent token is a logged denial, which is line #5.

The five gates

# Gate Refuses Config
1 Input guard prompt injection, jailbreaks, exfiltration prompts input_threshold, strict_input
2 Permission grants a call no live grant covers — checked four ways: grant exists, resource inside scope, not expired, budget unspent permission_gates, resource_args, exact_scope_tools
3 Approval gate malformed arguments, arguments carrying query syntax or instruction text, money above your limit — released only by the token the hold issued money_tools, order_id_pattern, refund_auto_max_cents, order_lookup
3b Per-task ceilings a task past its step or spend budget, and the same call repeated with identical arguments max_steps_per_task, max_usd_per_task, usd_per_step, loop_threshold
3c Deduplication (new in 1.3.0) a side effect that has already been performed — across tickets, requests, retries and worker processes, not just within one task dedup_dir, dedup_ttl_seconds, dedup_key_args
4 Reliability unbounded retries, no spacing between them, and hammering a dependency that is already down max_attempts, backoff_base, circuit_threshold, call_timeout
5 Output guard API keys, tokens, SSNs and card numbers on the way out redact_secrets
5b Read-only writes and mutating HTTP calls, for a tool meant only to read evidence — with gs.readonly():
Audit silent history: denials are recorded, timestamped, and the chain is tamper-evident audit_path

Gate 3c, the one this release adds

Through 1.2.3 this README said, under Honesty about what this is not, that nothing in this package deduplicates a side effect. That was true, it was the single most expensive thing on the list, and it is no longer true. It was also a contradiction: two of the free repos this package draws from advertise idempotency on their front pages — agent-reliability-kit ships dup-charges retry-only 5 / reliable 0, agent-approval-gate ships duplicate side effects 4 -> 0 — while the paid product denied it in four separate documents. Gate 3c is the answer to that contradiction. Both sides of it are checkable: the repos are public, and the old denials are quoted in place below rather than deleted.

python3 bench/dup_probe.py — offline, deterministic, four attempts at one $24.00 refund on one broken order:

  shape                         gate 3c off      on
  -----------------------------------------------
  one task, back to back             2 of 4    1 of 4
  four tickets, one order            4 of 4    1 of 4
  a lookup in between                2 of 4    1 of 4
  four workers                       4 of 4    1 of 4
  no task_id at all                  4 of 4    1 of 4
  -----------------------------------------------
  DUPLICATE refunds                      11       0
  money moved twice                 $264.00   $0.00

  40 concurrent OS processes, one identical refund: executed 1, deduplicated 39

Read the "off" column before the "on" column. The loop gate already caught half of two rows — that is gate 3b doing its job inside one task, in one process. The rows it cannot touch are the ones that are not one task and not one process: four tickets, four workers, and a call with no task_id at all. Those are 4 of 4, and they are the shape the incident actually took.

It is a file, and that is the whole mechanism. os.open(O_CREAT | O_EXCL) is atomic on POSIX, so of any number of processes racing to create one name exactly one wins. No daemon, no dependency, no network — the only kind of distributed lock this package is allowed to ship. Unset dedup_dir and the gate is off, so upgrading to 1.3.0 changes nothing until you turn it on.

And it is not free. It prevents the second execution; it does not return the first result. A worker that claims and then dies blocks the real refund until the TTL — a duplicate charge traded for a missed one. It needs one host or a shared filesystem. Claim files accumulate on disk and nothing sweeps them. All of that is priced in docs/LIMITS.md §9, and you should read it before you turn this on.

Before you trust any of this, read docs/LIMITS.md. It says what these gates do not do, with the measurements — including the one that matters most: the input guard blocks 27/27 of our corpus and 5 of 42 on the corpus written to break it. Both corpora are in the box, so you can reproduce the bad number in one command:

python bench/run_bench.py --corpus ./corpus/reviewer     # 5/42, 0 false alarms

No threshold setting closes that gap — the sweep is in docs/LIMITS.md §1. Gates 2, 3 and 5 are where this earns its keep.

Supported shape: one process, one writer. Grant budgets and expiry are per-process, and the audit log has no lock — four workers multiply every budget by four. guardstack check says so; docs/LIMITS.md §2 and §3 explain it.

Install

pip install -e ".[dev]"     # pytest + pyyaml + the adapters' deps
pytest -q                   # offline, no API key, ~3 s
PYTHONPATH=. python examples/support_agent.py

guardstack check guardstack.yaml    # will this config actually work?
guardstack verify ./guardstack-audit.jsonl   # has the log been tampered with?

In production, pip install guardstack — no dependencies at all. [dev] is only what step 1 above needs: pytest to run the suite, pyyaml so guardstack check can read the shipped .yaml (a .json config needs nothing), and fastapi/httpx for the adapter tests. A bare pip install -e . into a fresh virtualenv gives you a working library and no way to run the tests — which is a bad first five minutes, so it is named here.

The evidence, as a file you can hand to someone

python bench/run_bench.py --ablate --report evidence.md

Writes a dated, version-stamped, corpus-fingerprinted Markdown report: the headline table, what each permission gate alone prevents, and every case that went the wrong way, listed by name. That file is what you send to whoever asks "why is this safe?" — a terminal is not something you can attach to a review.

A finished one ships in this package: evidence-sample.md. It is that command's actual output against the corpus in corpus/ — dated, version-stamped, corpus-fingerprinted, and labelled section by section with what was measured on the shipped corpus and what was measured on synthetic fixtures. Read it before running anything: it is the clearest statement of what these gates do and do not do, and it is the file you would hand to a reviewer.

And so does the unflattering one: evidence-sample-adversarial.md — the same report against corpus/reviewer/, the corpus written to break this, where the input guard stops 5 of 42. Two reports, generated by the same command, shipped together. Shipping only the good one is how a measurement turns back into a claim.

The numbers here were measured on our corpus, which makes them a claim about our traffic. Point it at yours:

python bench/run_bench.py --corpus ./mycorpus --ablate --report evidence.md

corpus/TEMPLATE.jsonl is a starting point and docs/CORPUS.md is the format. Thirty lines from your own logs tell you more than the 65 we ship.

The report prices every gate, not only the permission ones. One dead dependency and twelve tasks that need it:

all gates on: reaches it 3 times, over 3s of backoff
  without breaker  reaches it 36 times — 33 more calls onto something already down
  without backoff  reaches it 3 times  — 3s of spacing removed; they arrive as
                                          fast as the loop can make them

Configure

Everything an integrator must change lives in one object:

# guardstack.yaml
input_threshold: 3          # lower = stricter
strict_input: false         # see docs/TUNING.md before turning this on

permission_gates: [grant, scope, expiry]
resource_args:              # which argument names the resource each tool touches
  issue_refund: scope       # every tool named below must appear here or in
  lookup_order: scope       # self_scoped_tools -- `guardstack check` enforces it
  cancel_order: scope
  send_email: to

audit_path: ./guardstack-audit.jsonl   # unset = memory only, gone on exit

money_tools: [issue_refund]
side_effecting: [issue_refund, send_email, cancel_order]
refund_auto_max_cents: 5000
refund_stale_after_days: 0   # 0 = off. Turning it on needs Config.order_lookup,
                             # which is a callable and so cannot live in YAML --
                             # the shipped file explains it. Note that commenting
                             # this key out does NOT turn the rule off: the
                             # default is 90.
approval_always: [cancel_order]

# gate 3c, new in 1.3.0: has this side effect already happened?
# Unset dedup_dir = off. Must be visible to every worker serving the same
# customers -- one host, or a shared filesystem. Claim files are never swept;
# prune them yourself. See docs/LIMITS.md §9 before turning this on.
dedup_dir:          /var/lib/guardstack/claims   # ABSOLUTE -- see below
dedup_ttl_seconds:  3600
dedup_key_args:              # what makes a call the SAME call. Without this
  issue_refund:     [order_id, amount_cents]   # every argument counts --
  cancel_order:     [order_id]                 # including a note the model
  send_email:       [to, scope]                # rewrites, which lets the
                                               # duplicate through.

# gate 3b, per task. Inert unless you pass check_tool(..., task_id=...), and
# task_id must be a value your model cannot choose -- see docs/LIMITS.md §9.
task_gates:         [budget, loop]
max_steps_per_task: 8
max_usd_per_task:   0.40
usd_per_step:       0.01     # without a price, the dollar ceiling is unreachable
loop_threshold:     3        # the same call+args 3x anywhere in the task.
                             # docs/TUNING.md prices it -- a workflow that
                             # legitimately re-reads one record gets stopped too.

max_attempts: 3

Copied as-is, this excerpt passes guardstack check with the one warning the shipped file also carries (per-worker budgets, §2 of LIMITS) and no others. It was checked, not eyeballed — an earlier version of it omitted the gate 3b block and therefore warned about an unreachable spend ceiling.

Two argument names are hardcoded and not in this object: money tools are read from an argument literally named amount_cents, and order_id_pattern applies only to one literally named order_id. If your tool calls the order id something else, that pattern check is skipped silently. QUICKSTART §2 has the table; docs/LIMITS.md §9 has the reproduction.

Config.from_file, Config.from_env (GUARDSTACK_*) and Config.from_dict all work. Unknown keys land in .extra instead of raising.

Three limits with no config key, and one thing with no limit at all

Three things that decide behaviour and are not in the object above. All three are caps, and each was added only after somebody measured the structure growing — _tasks in 1.2.1, _pending and the loop counter in 1.2.2.

  • Holds are capped at 1,024. Every call over the money limit opens a hold in memory. At 1,024 unanswered holds a new approval request is denied, not queued — refusing beats evicting, because dropping the oldest hold silently throws away a decision somebody is about to make. There is no expiry sweep, so the queue never drains on its own. Alarm on len(gs.pending_approvals).
  • The task table is capped at 4,096. Full means a new task_id is refused; tasks already being counted keep their budgets and keep working. Through 1.2.1 it evicted the oldest instead, and eviction handed a victim task's spent budget back at zero — a control bypass, fixed in 1.2.2. A full table means task ids are being minted per request, which turns all of gate 3b off anyway.
  • A task's loop counter is capped at 4,096 distinct call signatures. Beyond that, a new signature inside that task is refused at the loop gate with a reason that names max_steps_per_task — because a task making more than four thousand distinct calls is not one unit of work. This cap was written up as missing in 1.2.2's own release notes and then added before the release shipped; docs/LIMITS.md §9 keeps both halves of that story. audit_entries is the one structure inside a GuardStack still uncapped.

And gate 3c adds a second uncapped one, this time on disk. Every claim is a file in dedup_dir and there is no sweep — not in the library, not in guardstack check, not in the CLI. dedup_ttl_seconds bounds how long a claim blocks; it does not bound how many files exist. A file is removed only by gs.release(), or by a later claim on that same signature finding it stale. Measured, 1,000 distinct side effects:

files=1000  content=93709 B (93.7 B/claim)  allocated=4096000 B (4096 B/claim)

Read the second number: ~94 bytes of content, one 4 KiB filesystem block each, and one inode each. (The content column moves a few dozen bytes between runs — the claim body carries pid<N>@<host> — while files and allocated repeat exactly. Size against the allocated column.) At 10 distinct side effects/second for a day that is 864,000 files, 81 MB of content and 3.5 GB of allocated blocks in one directory. Operators must prune it themselves — a cron entry beside your log rotation, deleting claim files older than the TTL:

find /var/lib/guardstack/claims -name '*.claim' -mmin +60 -delete   # dedup_ttl_seconds: 3600

That is safe by construction: a claim older than the TTL is already breakable, so deleting it takes nothing away that the next claimant would not have taken. docs/LIMITS.md §9 has the full measurement.

A defect this README carried as open is closed. Until this revision the list above ended with a fourth item: "a non-string resource argument raises TypeError out of check_tool() — known, measured, and not fixed in 1.2.2". It is fixed: scope=['orders'], and every other non-str, non-None value, is now denied by the scope gate rather than escaping as an exception. Coercing resource arguments to str at your boundary is still tidy, but it is no longer load bearing. docs/LIMITS.md §9 has the finding, the fix and the re-run.

Honesty about what this is not

  • The detectors are pattern-based, not a model. They are fast, deterministic and auditable, and they will miss a novel phrasing. Gate 1 is a filter, not a proof.
  • Gate 4 still does not do idempotency by itself — gate 3c does, and only half of it. "Nothing in this package deduplicates a side effect." That sentence stood here through 1.2.3 and is now false. Left visible rather than deleted, because what this package sells is a record of what it was wrong about. What is true in 1.3.0:
    • execute()'s retries are still yours to make safe. With max_attempts=3 and a tool whose side effect lands and whose response is then lost, the body runs 3 times and gate 4 writes nothing to the audit log about any of them — unless you pass once=(tool, args), which takes one claim before the first attempt and holds it across all three.
    • execute(key=...) still does not help, and never did. That is the circuit-breaker key. once= is the dedup one.
    • Gate 3c prevents the second execution; it does not return the first result. A guard can refuse; it cannot invent the answer your provider gave the first time. For that you need your provider's own idempotency header, and gs.idempotency_key(tool, args) derives a stable one to pass it. That is a pure function and works with the gate off.
    • The claim lives in a file on one host or one shared filesystem, the key is only as stable as the arguments you name in dedup_key_args, and a worker that dies holding a claim blocks the real call until the TTL. docs/LIMITS.md §9 prices every one of those.
  • The numbers are measured on the shipped corpus, which is synthetic and hand-built. They tell you the gates work on the attacks I know about. Run bench/ against your own traffic before you quote them to anyone.
  • Gate 5 redacts known secret shapes. A secret with no shape — a plain password in prose — passes.
  • readonly() is a rehearsal aid, not a sandbox. io.open, raw os.open+os.write, os.truncate, os.chmod and anything that shells out go straight through. os.symlink and os.link did too until 1.2.2, and docs/LIMITS.md §7 did not list them — the table is re-measured there now.
  • One process, one writer, and this is the fourth place it bites. Grant budgets, the audit log, held approvals, and the task counters are all per-process and unlocked. Single-process asyncio is tested and safe; a thread pool sharing one GuardStack is not tested at all.

See NOTICE.md for what is free on GitHub, what the paid part actually is, the three defects found while assembling this, and — under "Defects found in our own documentation, in v1.2.2" — the two that were recorded as open and then fixed before the release shipped.

Licence

MIT. Yours to modify, deploy and resell.