ai-audit

client  --->  ai-audit  --->  api.openai.com
                 |            api.anthropic.com
                 |  scan, redact or refuse
                 v
            audit.jsonl
         hash chain + anchor

Local proxy for the OpenAI and Anthropic HTTP APIs. It scans each request for secrets and personal data, masks or refuses the call, and appends a hash-chained audit record.

git clone https://github.com/adagora/ai-audit.git && cd ai-audit && npm ci && npm run build && node dist/src/cli.js serve --mode redact --log audit.jsonl

There is no install script. This repository is not published on npm. npm install -g ai-audit installs a different program.

TL;DR

The problem. Agents and SDKs send prompts, tool results, screenshots and PDFs straight to the provider. A key or an IBAN in that payload is already gone. The record you can show later is whatever the provider kept.

The solution. Point the client at http://127.0.0.1:8787. ai-audit scans the body, applies one of three modes, forwards only what that mode allows, and writes one llm-call line to a JSONL file. ai-audit verify recomputes the hash chain. ai-audit anchor writes a digest you store apart from the log, so a swapped or shortened file shows up.

Why use ai-audit?

Behavior	What you get
redact	Text becomes [REDACTED:rule]. A hit in an image is blacked out. A hit in a PDF is rebuilt as page images.
no-go	HTTP 409. The provider is never called. The response lists rule ids.
advisory	The original body goes upstream. The finding is still in the log.
Fail closed	A torn log, a full disk, or an attachment the proxy cannot read stops the call.
Virtual keys	The client holds an aia- key. The provider key stays in the proxy process.
Your log	JSONL on disk. One record per inspected call. A daily anchor file kept somewhere else.
Overhead	About 0.6 to 0.7 ms p50 and about 1.1 ms p95 on a loopback mock, redaction included. See the 2026-10-20 run.

TypeScript, Node's standard library, no runtime dependencies. Quinelings is optional and is not on the request path. Decisions are in docs/adr.

What it looks like

The operator console over the synthetic log of npm run example:workflow and npm run example:wycena -- --console --wait. No real traffic.

What needs attention now, and what the log suggests changing:

The state machine of a request, with how many requests reached each state and where they stopped:

What is in the context of one request: each block in prompt order, its kind, size and age, and what made it large. The log stores sizes, never text:

Decisions and their verdicts: the rule's result, the person's verdict and what the application did after it:

Quick example

For one local Codex session, run npm run codex:proxy from the repo root. It builds the proxy, asks for your OpenAI API key if AI_AUDIT_UPSTREAM_OPENAI_KEY is unset, generates a temporary aia- client key, starts the proxy on 127.0.0.1:8788, and launches Codex through it. The provider key stays out of Codex's environment; the proxy stops when Codex exits. The audit log and proxy diagnostics stay in ignored live/. Set AI_AUDIT_PORT=8787 if you have stopped the existing demo proxy and want the native Grafana scrape to follow this session. This launcher starts the proxy and Codex, not Grafana.

git clone https://github.com/adagora/ai-audit.git
cd ai-audit
npm ci && npm run build

keygen_out=$(node dist/src/cli.js keygen --name agent)
client_key=$(printf '%s\n' "$keygen_out" | sed -n 's/^key (give to the client, not stored): //p')
config_entry=$(printf '%s\n' "$keygen_out" | sed -n 's/^config entry: //p')
printf '%s\n' "{\"keys\":[$config_entry]}" > gateway.json
printf 'export AI_AUDIT_CLIENT_KEY=%s\n' "$client_key"

export AI_AUDIT_UPSTREAM_OPENAI_KEY="sk-your-provider-key"
node dist/src/cli.js serve --mode redact --log audit.jsonl --config gateway.json

serve stays in the foreground. In a second shell, paste the export line the first shell printed, then:

curl -sS http://127.0.0.1:8787/healthz
curl -sS http://127.0.0.1:8787/v1/chat/completions \
  -H "authorization: Bearer $AI_AUDIT_CLIENT_KEY" \
  -H "content-type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"mail ada@example.com"}]}'

node dist/src/cli.js verify --log audit.jsonl
node dist/src/cli.js stats --by model --log audit.jsonl
node dist/src/cli.js anchor --log audit.jsonl --anchors anchors.jsonl

With virtual keys configured, the provider sees [REDACTED:pii-email] in place of ada@example.com. The client still receives the model's answer. The response carries x-request-id, which is the id of the llm-call row, and x-audit-finding-count.

Claude Code and the Anthropic SDK:

ANTHROPIC_BASE_URL=http://127.0.0.1:8787 claude

The OpenAI SDK can use OPENAI_BASE_URL. For Codex CLI, the shortest supported URL override is codex -c 'openai_base_url="http://127.0.0.1:8787/v1"'. That uses Codex's existing OpenAI credential. It works with the proxy's credential-pass-through mode, where no virtual key is configured and the client holds the provider credential. With the recommended aia- virtual key, select an explicit custom Responses provider so Codex sends that key to the proxy:

export AI_AUDIT_CODEX_KEY="$AI_AUDIT_CLIENT_KEY"
codex -m gpt-6-sol \
  -c 'model_provider="ai_audit"' \
  -c 'model_providers.ai_audit.name="ai-audit"' \
  -c 'model_providers.ai_audit.base_url="http://127.0.0.1:8787/v1"' \
  -c 'model_providers.ai_audit.env_key="AI_AUDIT_CODEX_KEY"' \
  -c 'model_providers.ai_audit.wire_api="responses"' \
  -c 'model_providers.ai_audit.requires_openai_auth=false' \
  -c 'model_providers.ai_audit.supports_websockets=false'

The OpenAI API key remains in the proxy process as AI_AUDIT_UPSTREAM_OPENAI_KEY. Codex sends only the aia- virtual key. The proxy supports /v1/responses; gpt-6-sol must be available to your API key. These flags apply to this Codex invocation and do not change your normal Codex configuration. Codex provider settings.

Optional request headers x-audit-system-id and x-audit-use-case are copied onto the log record as labels. With a use-case table, the use case also selects limits.

For a browser-driven integration test with your own OpenAI API key, run npm run example:workflow:real and follow the workflow cockpit guide. The same guide shows how to point UniiChat at the running proxy. Use npm run example:workflow for deterministic mock-provider scenarios.

Design

Scan the whole request, then decide once. Text, tool arguments, tool results, nested JSON, images and PDFs are inspected before anything is forwarded. The decision is forward-original, forward-redacted, or block. ADR 0009.

Stop when the evidence cannot be kept. If the log is torn or cannot be appended, the proxy returns 503 and stops before the upstream call. count_tokens is scanned and never logged. ADR 0016.

The log stores a rule id and a JSON path. Each pair is stored once with a count, and at most 1,000 pairs per record; further matches are counted per rule in findingsOverflow (ADR 0051). Response bodies are stored as outputHash unless that virtual key opted into redacted text. ADR 0007, ADR 0011.

Provider keys stay in the proxy environment. keygen prints an aia- key. Config stores its SHA-256. The upstream call uses AI_AUDIT_UPSTREAM_OPENAI_KEY or AI_AUDIT_UPSTREAM_ANTHROPIC_KEY. ADR 0008.

A decision is a later record. ai-audit decide links a policy result to x-request-id. Replay reads the log. The proxy still serves when the Quinelings package is absent. ADR 0004.

Requests and decisions

There are two kinds of decision, and only the first one can stop a request.

• The proxy decides about each request, before anything is forwarded: a key, a use case, the request shape, the model and the scan can refuse it, and a no-go finding blocks it. The provider never sees a stopped request. After the answer, the tool gate checks the tool calls the model proposed. One llm-call or refusal record is written.
• A rule decides about the work after the model has answered, in your application. It does not stop the request; the answer is already back. It decides whether the action that follows, such as sending a quote, may run now or waits for a person. The application waits with await and records what it did with effect.

All of these are records in the same hash-chained log: llm-call, refusal, decision with its :qdl step, program, review and effect. The console reads them from there.

sequenceDiagram
  participant App as Application
  participant Proxy as ai-audit proxy
  participant LLM as Provider
  participant Rule as QDL rule
  participant You as Reviewer in the console
  App->>Proxy: request, with key and use case
  Note over Proxy: may refuse or block here, then nothing is forwarded
  Proxy->>LLM: forwarded request, redacted if needed
  LLM-->>Proxy: answer
  Note over Proxy: tool gate checks proposed tool calls
  Proxy-->>App: answer, llm-call record
  App->>Rule: decide with the model's numbers
  Rule-->>App: result and proposed action, decision record
  App->>App: await verdict
  You-->>App: approve, override or reject, review record
  App->>App: run or skip the action, effect record

Loading

The request path, in the order the proxy checks it. The console's Request flow page counts each stage (STAGES in src/consoleModel.ts). This chart is simplified: it leaves out the log and the disk failing, which the request lifecycle model covers in 148 states:

stateDiagram-v2
  received : Request received
  key : Virtual key
  useCase : Use case
  shape : Route and body
  model : Model allowed
  inspect : Inspection
  decide : Decision
  budget : Token budget
  upstream : Provider
  gate : Tool gate
  recorded : Delivered and recorded
  refused : Refused 400, 401, 403, 404 or 413
  rejected : Rejected 422
  blocked : Blocked 409
  budgetRefused : Budget refused 429 or 503
  noAnswer : No provider answer 408 or 502
  [*] --> received
  received --> key
  key --> useCase
  useCase --> shape
  shape --> model
  model --> inspect
  inspect --> decide
  decide --> budget
  budget --> upstream
  upstream --> gate
  gate --> recorded
  recorded --> [*]
  key --> refused: unknown key or provider
  useCase --> refused: missing, not allowed or off
  shape --> refused: route, size or JSON
  model --> refused: model not on a list
  inspect --> rejected: deadline or uninspectable part
  decide --> blocked: finding in no-go mode
  budget --> budgetRefused: budget used up
  upstream --> noAnswer: deadline or network
  note right of gate : enforce mode rewrites the answer when a call is held

Loading

A decision after the answer, drawn with /statechart from spec/DecisionLoop.tla (docs/verification/decision-loop.cfg, 52 model states). A decision takes one verdict, an application checks for a verdict before an action the rule allows, and a wait that times out records skipped. The two red states are the known windows of ADR 0057: an approval that comes after the application skipped, and a verdict that comes after its check. The console's Decisions page shows each decision as waiting, acted, approved, overridden or rejected.

---
title: "A decision after the answer: rule, verdict, action"
---
stateDiagram-v2
  state "Legend" as legend {
    lg0: decide
    lg1: act
    lg2: end
    lg3: Breaks ApprovalIsActedOn or NeverFlagged
    lg4: Path to the ApprovalIsActedOn or NeverFlagged bug
  }
  class lg0 blue
  class lg1 yellow
  class lg2 teal
  class lg3 breaks
  class lg4 bugPath
  s0: Decision recorded
  s1: App about to act
  s2: Waiting for a person
  s3: Verdict recorded, app not told yet
  s4: App about to skip
  s5: Action done, not recorded yet
  s6: Done and authorized
  s7: Skipped
  s8: Skipped, then approved
  s9: Done, flagged as without approval
  [*] --> s0
  s0 --> s2: the rule asks for a person
  s0 --> s1: the rule allows the action and no person said no
  s0 --> s3: the rule asks for a person
  s0 --> s4: the rule allows the action, but a person said no first
  s1 --> s5: the app acts
  s2 --> s3: a person records approved or a person records taken over or a person records rejected
  s2 --> s4: the wait times out
  s3 --> s1: the app reads approved
  s3 --> s4: the app reads rejected or taken over
  s4 --> s4: a person records approved
  s4 --> s7: the app records skipped
  s5 --> s6: the app records done
  s5 --> s5: a person records taken over
  s4 --> s8: the app records skipped
  s5 --> s9: the app records done [a verdict came after its check]
  s7 --> s8: a person records approved
  classDef breaks fill:#ffecee,stroke:#e72e3b,stroke-width:2.5px,color:#cd2834
  class s0 blue
  class s0 bugPath
  class s1 yellow
  class s1 bugPath
  class s2 blue
  class s2 bugPath
  class s3 blue
  class s4 yellow
  class s4 bugPath
  class s5 yellow
  class s5 bugPath
  class s6 teal
  class s7 teal
  class s8 teal
  class s8 breaks
  class s8 bugPath
  class s9 teal
  class s9 breaks
  class s9 bugPath

Loading

The formal model of the request lifecycle, with log failures, is in docs/verification.

How it compares

	ai-audit	Call the provider directly	LiteLLM proxy	gitleaks on the repo
Runs on the live API call	yes	the call is the API	yes	no, it scans files
Masks or blocks a secret in the prompt	yes, before forward	no	only with a guardrail you add	no
Hash-chained JSONL plus an anchor file you keep	yes	no	no	no
Image and PDF inspection in a sandbox	yes, Linux	no	no	no
Virtual keys, provider key stays on the proxy	yes	no	yes	no
One OpenAI-shaped API for many providers	no	no	yes	no
Setup	clone, npm ci, build	one environment variable	a gateway config	a CLI in CI

Use ai-audit when Claude Code, Codex, or an official SDK must be scanned on the way out, and you want a log you can re-hash after the fact.

Use LiteLLM when the job is to send one client format to many providers. This proxy does not translate OpenAI into Anthropic. An alias may change the model or the base URL inside the same API. ADR 0006.

Use gitleaks when the secret is in a file in git. It does not see the tool result an agent is about to post.

A hosted logging gateway keeps prompts on that service. ai-audit keeps the JSONL on the machine that runs the proxy. Response text is omitted unless the virtual key set logContent to true, and even then the stored text is redacted.

Installation

Node 22.23.4 or later on the 22 line, 24.22.1 or later on the 24 line, or 26.12.1 or later. CI runs the tests on Node 22, 24 and 26. There is no curl installer, no Homebrew formula, and no package on npm under this name.

From source

git clone https://github.com/adagora/ai-audit.git
cd ai-audit
npm ci
npm run build
node dist/src/cli.js serve --mode redact --log audit.jsonl

npm ci installs the vendored Quinelings SDK from vendor/. The proxy starts without that package. decide, replay and policies then fail with audit_qdl_unavailable.

On your PATH

npm run build
npm link
ai-audit serve --mode redact --log audit.jsonl

npm link points at dist/src/cli.js. Build first. From the repo directory, npx ai-audit serve --mode redact --log audit.jsonl uses the same bin.

Docker

The image listens on 0.0.0.0:8787 and writes /data/audit.jsonl. The default base is node:26.12.1-slim.

docker build -t ai-audit .
docker run --rm -p 8787:8787 -v "$PWD/data:/data" \
  -e AI_AUDIT_UPSTREAM_OPENAI_KEY \
  -e AI_AUDIT_CLIENT_KEY \
  ai-audit

Extra arguments replace the image command. To change the mode, pass the full serve line:

docker run --rm -p 8787:8787 -v "$PWD/data:/data" \
  -e AI_AUDIT_UPSTREAM_OPENAI_KEY \
  ai-audit serve --host 0.0.0.0 --log /data/audit.jsonl --mode no-go

Attachment workers use bubblewrap. Inside Docker that needs extra privileges, or every attachment returns 422 audit_sandbox_unavailable:

docker run --rm -p 8787:8787 -v "$PWD/data:/data" \
  --cap-add SYS_ADMIN \
  --security-opt seccomp=unconfined \
  --security-opt apparmor=unconfined \
  --security-opt systempaths=unconfined \
  -e AI_AUDIT_UPSTREAM_OPENAI_KEY \
  ai-audit

Pointer-compressed builds are opt-in and slower for this process. Measured with node bench/memory.mjs, five runs: JS heap 6.3 MB instead of 9.7 MB, RSS still about 180 MB, throughput about 8% lower. The default image stays official Node. ADR 0005.

docker build --build-arg BASE=platformatic/node-caged:26.12.1-slim -t ai-audit .

Quick start

1. Build, as in From source.

2. Create a client key and save the printed config entry.

   node dist/src/cli.js keygen --name agent

3. Write gateway.json with that sha256. A full file is in Configuration.

4. Export the provider key into the proxy's environment, not the client's.

   export AI_AUDIT_UPSTREAM_OPENAI_KEY="sk-your-provider-key"
   export AI_AUDIT_ADMIN_KEY="choose-a-long-admin-secret"

5. Start the proxy.

   node dist/src/cli.js serve --mode redact --log audit.jsonl --config gateway.json

   stderr prints one JSON line: event is listening, plus the URL, mode, log path, sandbox state and key count.

6. Point the client at the proxy and send the aia- key as Authorization: Bearer or x-api-key.

7. Check the chain, then store an anchor off the same disk as the log.

   node dist/src/cli.js verify --log audit.jsonl
   node dist/src/cli.js anchor --log audit.jsonl --anchors /elsewhere/anchors.jsonl

With an empty keys array and no AI_AUDIT_CLIENT_KEY, the proxy forwards the client's Authorization or x-api-key unchanged. The request is still scanned. The client is holding the provider key.

Optional employee token budgets

For an OpenAI monthly spending cap, use its organization or project hard spend limit. Several official keys in one project share that project's allowance. Our optional quota is useful for separate employee token allowances in a shared project or across OpenAI and Anthropic, with every request passing through inspection.

Add tokenBudget to the employee's existing key entry in gateway.json:

{
  "budgetDirectory": "live/employee-budgets",
  "keys": [{
    "name": "anna",
    "sha256": "paste-the-sha256-keygen-printed",
    "models": ["*"],
    "tokenBudget": { "period": "month", "inputTokens": 1000000, "outputTokens": 100000, "maxOutputTokens": 4096 }
  }]
}

day and month reset at UTC calendar boundaries. The proxy counts inspected input at the provider, reserves that input and the capped output before generation, then settles reported usage. Missing usage keeps its full reservation. Concurrent processes using the same directory cannot reserve the same balance. Exhaustion returns 429 audit_token_budget_exceeded. Keep the directory across restarts and log rotation; deleting it or changing key hashes creates a new allowance. Requests still in flight at rollover belong to their admission period.

Budgeted keys support OpenAI Responses and Anthropic Messages, including streaming. They add a provider count request and set or clamp max_output_tokens / max_tokens to maxOutputTokens. Chat Completions, stateful Responses references, background execution and provider-executed tools return 422 for budgeted keys. Keys without a budget retain their existing routes. These token quotas are not dollar budgets: cached tokens count toward input, while provider pricing and non-token charges differ.

Counting or accounting failures refuse generation with 503. A provider count can differ from final usage; any reported overrun is charged fully and blocks that key until its next period. This cannot undo provider-side overspending on that request. Crash reservations remain charged conservatively, and abandoned lock files require investigation while writers are stopped. State is intended for a shared local filesystem, not separate hosts or network filesystems. ADR 0044.

Inspect current allowances with the admin credential:

curl -H "x-admin-key: $AI_AUDIT_ADMIN_KEY" http://127.0.0.1:8787/audit/budgets

The response lists each budgeted key's name, limits, spent, reserved, remaining and blocked status. Request records also carry the budget policy, reservation and settlement status. Accounting files contain no prompts, responses or raw credentials and must stay out of Git.

Commands

After npm link, the binary is ai-audit. The examples below use that name. node dist/src/cli.js is the same program.

Command	Example	Effect
serve	ai-audit serve --mode redact --log audit.jsonl --port 8787	Listens. Default host is 127.0.0.1.
console	ai-audit console --log audit.jsonl --config gateway.json --reviewer ada	Opens the operator console on 127.0.0.1:8793. See Operator console.
verify	ai-audit verify --log audit.jsonl	Recomputes the chain, and accepts a line only byte for byte as ai-audit wrote it (ADR 0058). Exit 0 when it matches, 1 when it does not.
verify	ai-audit verify --log audit.jsonl --anchors anchors.jsonl	Also catches a swapped or truncated file.
anchor	ai-audit anchor --log audit.jsonl --anchors anchors.jsonl	Appends an anchor. Run it daily. Keep the anchor file somewhere else.
rotate	ai-audit rotate --log audit.jsonl	Closes the log into a timestamped segment and starts a new file linked to it.
prune	ai-audit prune --log audit.jsonl --keep-days 184	Records, then deletes, closed segments whose last record is older than N days.
report	ai-audit report --log audit.jsonl --from 2026-10-01 --to 2026-11-01	Markdown on stdout for that interval.
stats	ai-audit stats --log audit.jsonl --by key	Table of tokens, cached tokens, latency and errors. --by is key, model or hour.
context	ai-audit context --log audit.jsonl	Table of sessions.
context	ai-audit context --log audit.jsonl --session ID --output session.html	One self-contained HTML page.
workbench	ai-audit workbench --log audit.jsonl --output workbench.html	Private, self-contained index of sessions, context sizes, scan history, Quinelings observations and decisions from a verified log snapshot.
keygen	ai-audit keygen --name agent	Prints the client key and a config object. Add --log-content to store redacted output for that key.
profile	ai-audit profile --mode redact --config gateway.json	JSON identity of the current scanner, mode and config.
scrub	ai-audit scrub --input history.jsonl --output history.redacted.jsonl	A separate sanitized copy, plus a receipt on stdout.
check-quotes	ai-audit check-quotes --source email.txt --answer answer.json	Checks values the model claims to have read.
decide	ai-audit decide --policy policy.json --inputs inputs.json --system-id shop --use-case quote	Records a policy decision.
approve	ai-audit approve --decision ID --reviewer ada --verdict approved	Appends a review. Verdicts are approved, overridden, rejected. A decision takes one verdict; a second one is refused with audit_already_reviewed (ADR 0057).
replay	ai-audit replay --decision ID --anchors anchors.jsonl	Recomputes that decision from the log.
await	ai-audit await --decision ID --timeout-s 600	Waits for a review of that decision. Exit 0 approved, 1 rejected or overridden, 3 timeout. With --timeout-s 0 it only checks; run it before an action the rule allows, and record skipped with code timeout when you stop waiting.
effect	ai-audit effect --decision ID --action send-quote-to-customer --status done --review REVIEW_ID	Records that the application ran, skipped or failed an action of that decision.
lab	ai-audit lab --log audit.jsonl --session ID --baseline-limit 2 --candidate-limit 3 --output comparison.html	Compares failure thresholds on recorded observations; never stops an agent.
receipt	ai-audit receipt --decision ID --output receipt.html	HTML page with downloads for a completed QDL decision. --format svg or --format json exports a portable receipt.
verify-receipt	ai-audit verify-receipt --input receipt.svg	Replays a portable JSON or SVG receipt without the original log. Either form verifies only exactly as ai-audit wrote it (ADR 0059).
attest-keygen	ai-audit attest-keygen --private-key reviewer.pem --public-key reviewer.pub.pem	Creates an Ed25519 signing key pair in new files.
attest	ai-audit attest --log audit.jsonl --decision ID --verdict approved --private-key reviewer.pem --output decision.attestation.json	Signs the replayable decision, latest matching review record and log checkpoint.
verify-attestation	ai-audit verify-attestation --input decision.svg --attestation decision.attestation.json --public-key reviewer.pub.pem	Replays the receipt and verifies the detached signature with the supplied key.
policies	ai-audit policies --system-id shop --use-case quote	One JSON object per policy version on that use case.

Share a QDL decision

ai-audit receipt --log audit.jsonl --decision ID --output receipt.html
ai-audit receipt --log audit.jsonl --decision ID --format svg --output decision.svg
ai-audit receipt --log audit.jsonl --decision ID --format json --output decision.json
ai-audit verify-receipt --input decision.svg

The HTML page includes SVG and JSON downloads for a completed, consistent QDL decision. The SVG shows the Quinelings body beside its calculation steps and embeds the exact program and inputs in XML metadata. You can place it in a report or send it to someone who has the compatible Quinelings SDK. The SVG, JSON and HTML output paths must be new files; they never replace the source log.

verify-receipt checks that the program can be recovered and that the recorded calculation runs again. For SVG, it also checks that the visible drawing matches the embedded data. It returns origin: "not-verified" and humanApproval: "not-verified": a recipient cannot authenticate the log, its external anchor, the author or a human review from this file alone. Keep the original log and an independently stored anchor for those checks. A PNG or screenshot loses the replay metadata. ADR 0036.

Sign a reviewed decision and checkpoint

ai-audit attest-keygen --private-key reviewer.pem --public-key reviewer.pub.pem
ai-audit approve --log audit.jsonl --decision ID --reviewer "Ada" --verdict approved
ai-audit receipt --log audit.jsonl --decision ID --format svg --output decision.svg
ai-audit attest --log audit.jsonl --decision ID --verdict approved --private-key reviewer.pem --output decision.attestation.json
ai-audit verify-attestation --input decision.svg --attestation decision.attestation.json --public-key reviewer.pub.pem
ai-audit workbench --log audit.jsonl --output signed-workbench.html --input decision.svg --attestation decision.attestation.json --public-key reviewer.pub.pem

Keep the private key outside the log and backups of the log. Give the public key to recipients through a channel they already trust. The attestation binds the portable receipt to one signed checkpoint and the latest recorded review verdict; it omits the reviewer name and note. Verification authenticates the key and signed review record, not the person's identity, the truth of the review, or a trusted timestamp. If the key is not independently trusted, the signature only proves internal consistency. The workbench checks that its verified log contains the signed checkpoint before displaying the status. ADR 0038.

Review before sharing. The portable files include exact policy source, inputs and outputs. Export applies the scanner, but that is not complete anonymisation. The full HTML receipt can also contain reviewer names, notes and caller assertions. Refused, failed and incomplete decisions remain diagnostic HTML; they cannot become a portable replay receipt.

--log defaults to ./audit.jsonl on every command that reads the log. --from defaults to 1970-01-01 and --to defaults to 9999-12-31. anchor with no --anchors writes <log>.anchors beside the log. Keep that file somewhere else if it is supposed to catch a replaced log.

serve

ai-audit serve \
  --mode redact \
  --log audit.jsonl \
  --host 127.0.0.1 \
  --port 8787 \
  --config gateway.json \
  --allow-hashes allow.json \
  --upstream-openai https://api.openai.com \
  --upstream-anthropic https://api.anthropic.com \
  --rotate-bytes 268435456

--rotate-bytes closes the log into a segment before an append once the file is larger than that, default 256 MiB, 0 to never rotate. See Rotation and retention.

A request whose text adds up to more than 256 KiB is scanned in a worker thread, so other requests are not held while it runs. --inspect-timeout-ms, default 5000, is its deadline. A scan that takes longer is stopped, the request is refused with 422 audit_inspection_timeout and is not forwarded. At most min(4, CPUs - 1) workers run; when 64 more requests already wait, the next one gets 503 audit_inspection_busy. Ordinary text scans at about 26 ms per MiB on an M1; some repeated patterns take up to 0.74 s per MiB. ADR 0050.

--mode is redact, no-go or advisory. The default is redact. --no-page-redaction and "pageRedaction": false both turn PDF rasterizing off. A PDF finding then returns 409 instead of a rebuilt file. ADR 0013.

--unsafe-no-sandbox, or AI_AUDIT_UNSAFE_NO_SANDBOX=1, runs attachment workers with no bubblewrap. Development only. The log records decision.sandbox as disabled. ADR 0021.

Request	Who can call it	Result
GET /healthz	anyone	ok
GET /readyz	anyone	ready, or 503 audit_tools_missing, or 503 audit_sandbox_unavailable
GET /metrics	anyone	OpenMetrics text. Counts only.
GET /audit/stats?by=model	header x-admin-key	Same numbers as ai-audit stats
GET /v1/models, GET /v1/models/:id	virtual key, when keys are configured	Passed through, not scanned as a prompt
POST /v1/chat/completions	virtual key, when keys are configured	Scanned, then forwarded or refused
POST /v1/responses	same	same
POST /v1/messages	same	same
POST /v1/messages/count_tokens	same	Scanned, not written to the log

Anything else is 404 audit_unsupported_route.

Bodies over 32 MiB return 413 audit_body_too_large. Bad JSON is 400 audit_malformed_json. Duplicate keys are 400 audit_duplicate_keys. Nesting deeper than 256 is 400 audit_json_too_deep. The upstream call is abandoned after 180 seconds with 408 audit_deadline_exceeded. An upstream that cannot be reached is 502 audit_upstream_unreachable. Both of those are still logged.

Error bodies look like this. On a block, findings holds rule ids and nothing else.

{"type":"error","error":{"type":"audit_proxy_error","code":"audit_findings_blocked","message":"audit_findings_blocked","findings":["pii-email"]}}

Mode	A finding becomes
redact	[REDACTED:rule] in text. Images blacked out. PDFs rebuilt as page images, logged as decision.rasterized.
no-go	409 audit_findings_blocked. Upstream is not called.
advisory	Original bytes forwarded. Finding logged.

A finding that cannot be masked is 409 audit_unredactable_finding in redact and no-go. That includes a secret in an object key, nested duplicate keys, and a box the PDF rasterizer cannot map. The rebuilt image or PDF is scanned again. A hit that survives is also 409.

A rasterized PDF reaches the model as page images. There is no text layer. The call spends more tokens, and the model is reading OCR.

Prompt injection uses deterministic English and Polish rules on tool results and attachment text, including Polish without diacritics and screening of normalized Unicode. The proxy logs the rule and the path, sets x-audit-signal-count, and still forwards when the mode would forward. Clients resend earlier tool results on each turn, so every record lists the match again; /metrics, stats and report count a rule at a path one time per session. Matches are advisory even in no-go mode: quoted examples also match, and paraphrases can evade the rules. See detection limits, model candidates and blocking criteria and ADR 0010.

An optional local Ollama judge adds semantic observations on tool-result text: --injection-model qwen3.5:4b --injection-task 'Describe the authorized task here'. It examines the newest tool results first, reuses a verdict for text it has already judged, remains advisory and reports incomplete inspection explicitly. Run npm run example:injection -- --ollama to compare it with the regex baseline on synthetic English/Polish/mixed-language cases. See setup and evaluation and ADR 0045.

Tool-call gate

Text checks are too slow or too inaccurate to block on, so blocking belongs at the action. actionGate in gateway.json lists dangerous tools by exact name, their sensitive arguments and the destinations they may use:

"actionGate": { "tools": {
  "send_email": { "mode": "observe", "arguments": ["$.to", "$.cc"], "allow": ["@company.example"] },
  "http_post": { "mode": "observe", "arguments": ["$.url"], "allow": ["*.company.example"] }
} }

For each tool call the model proposes to a listed tool, the proxy takes URL hosts, email addresses, phone numbers and IBANs from those arguments. A destination passes if it is allowed or the user typed it. A destination that came only from a tool result, a tool definition or the system prompt, or from nowhere, is held. A secret in the arguments is always held. No model runs, so the check costs microseconds and does not depend on language or wording.

observe delivers the response unchanged and records actionGate.calls: per value its argument path, kind, origin (user, system, untrusted, none, unknown), the source path of an untrusted occurrence, and the result. Values are never stored. Unlisted tools pass, and tools the provider runs itself are not seen. ADR 0046.

enforce blocks. If one call in a response is held, the client receives none of that response's tool calls. In their place it gets a text answer, [ai-audit] audit_action_held: held tool call send_email. …, with a normal stop reason, so the agent can tell the user and continue. This works for Chat Completions, Responses and Messages, streamed or not. A streamed answer is delivered at once up to its first tool-call event; the rest waits for the end of the response. The record has actionGate.rewritten: true. Requests without an enforce rule take the unchanged pass-through path. In redact mode, email addresses, phone numbers and IBANs in tool results are masked before the model sees them, so the gate matters most for URL hosts and in advisory mode.

A virtual key can also limit which tools its model may call: "tools": { "mode": "observe", "allow": ["propose_quote", "mcp__crm__*"] } in its keys[] entry. A call to any other tool is held and recorded with toolAllowed: false. Tool definitions in the request are not changed, so clients that send all their tools keep working. ai-audit report lists the tools each key called, so you can build the list from real use. A key without tools is not limited. ADR 0048.

Use cases

A use case is one kind of work that an application gives to a model, such as summarizing a ticket or drafting a refund. The application knows which of its features makes the call, so the client names the use case in the x-audit-use-case header. The proxy never guesses it from the content. With a useCases table in gateway.json, each use case gets its own limits:

"useCases": {
  "summarize-ticket": { "tier": "assist", "models": ["gpt-4o-mini"] },
  "refund-draft": { "tier": "shadow", "tools": { "mode": "enforce", "allow": ["crm_lookup"] } },
  "legacy-export": { "tier": "off" }
},
"keys": [{ "name": "support-app", "sha256": "…", "useCases": ["summarize-ticket", "refund-draft"] }]

Field	Meaning
tier	off, shadow, assist, contain or transact: how much the application lets the model do without a person. The proxy refuses off with 403 audit_use_case_off. It records the other tiers but cannot enforce them, because it does not see whether a person reads the answer.
models	Model ids allowed for this use case, in addition to the key's own list.
tools	mode and allow, as in a key's tools. A call outside this list is held and recorded with useCaseToolAllowed: false.
keys[].useCases	The use cases a key may name. A key without the list may name every row. If the header is missing and the list has one name, that name applies.

With a table, a request that names no row gets 400 audit_use_case_required, and a row that is not in the table or not in the key's list gets 403 audit_use_case_not_allowed. Each llm-call record has useCase and useCasePolicy with the tier and a digest of the row, so a later change to a row is visible. ai-audit report and the workbench show calls, refusals, held tool calls, tokens, decisions and reviews per use case. A decision recorded with decide --use-case NAME is counted with the same row. Without a table, the header is only a label, as before. ADR 0053.

In Claude Code, send the header with ANTHROPIC_CUSTOM_HEADERS='x-audit-use-case: coding', or give it a key whose list has one row.

What the scanner matches

Rules are adapted from Gitleaks, plus the rows below. A match followed by punctuation still counts. A secret split across text parts still counts. OCR text, PDF text and PDF metadata are scanned. JSON inside tool arguments and tool results is walked. See THIRD_PARTY_NOTICES.md and ADR 0028.

Rule ids	What matches
aws-access-token, aws-secret-access-key, aws-session-token	AWS key id shapes, a secret next to a key id, an unlabeled KEY_ID : SECRET pair, a percent-encoded secret, a session token
github-pat, github-fine-grained-pat, github-app-token, github-oauth, github-refresh-token	ghp_, github_pat_, ghu_, ghs_, gho_, ghr_
openai-api-key, anthropic-api-key, anthropic-admin-api-key, openrouter-api-key, xai-api-key	Those vendors' key shapes
gcp-api-key, stripe-access-token, slack-bot-token, slack-user-token, slack-webhook-url	GCP, Stripe, Slack
azure-ad-client-secret	Azure AD client secret shape, and AZURE_CLIENT_SECRET by name
jwt, private-key	JWTs, including wrapped ones, and PEM private keys
presigned-url-credential	X-Amz-Security-Token, X-Amz-Signature, X-Goog-Signature
database-credentials, database-password	URLs and Go user:pass@tcp(...) DSNs with credentials. The whole URL or DSN is masked. PGPASSWORD, MYSQL_PWD, db_password, including prefixes such as PROD_DB_PASSWORD. Userinfo passwords and password, passwd, pwd query parameters, including percent-encoded names
credential-client-secret, credential-refresh-token	OAuth client secrets and refresh tokens
pii-email, pii-phone, pii-pesel, pii-iban	Email addresses. Phone numbers written with + or 00, or as three groups of three digits. PESEL. IBAN with checksum and country length, including a Polish IBAN written without the PL prefix
named-credential	A value assigned to, or a JSON field named, *password, *secret, *token, *private_key, *api_key or *access_key. Code references, types and pagination tokens are skipped. A GCP service-account private_key is in this set
ambiguous-json, deep-json	Nested JSON that cannot be rewritten safely. Enforcing modes block

--allow-hashes points at a JSON array of lowercase SHA-256 hex digests of exact secret values. A listed value is not a finding. The digest is not written to the log.

Every logged decision carries inspectionProfile: scannerVersion, rulesDigest, configDigest, and a combined digest. The same snapshot keys the attachment cache. Older rows stay valid without a profile. Their policy cannot be rebuilt later. ADR 0014.

ai-audit profile --mode redact --no-page-redaction --allow-hashes allow.json

Sessions, usage and the context map

A request whose message list starts with an earlier request's full list is the next turn of that session. An edited history branches at the last shared turn. Digests are HMACs under a key in <log>.session-key, mode 0600. The HMAC input is the message list. ADR 0018.

Each llm-call has context.bytes for system, tools, user, assistant, toolResults, attachments and other. When the provider reported prompt tokens, context.tokens splits that total by size and tokenBasis is provider-scaled. Otherwise tokenBasis is unknown. Sizes only. ADR 0017.

A call the provider did not price is counted in unknownUsage, per field unknownInputUsage and unknownOutputUsage. It is never stored as zero. tokensComplete: false means the totals are a lower bound. ADR 0012.

OpenAI chat streams omit usage unless the client sends "stream_options": {"include_usage": true}. The proxy does not add that flag.

ai-audit context --log audit.jsonl
ai-audit context --log audit.jsonl --session "$SESSION_ID" --output session.html

$SESSION_ID is an id from the table printed by the first command. An unknown id prints audit_unknown_session and exits 2.

The HTML page has two charts. The timeline is provider-reported cache read, cache write, uncached input and output. A cache bust is marked P when the prefix was rewritten and C when the provider dropped the cache. The arena is one row per request, blocks in prompt order, colored by how many turns a block stayed at that position. A red marker is the first changed offset. Cached tokens after that offset are gone. The header shows peak tokens, cache hit rate, busts, and the share taken by tools and system. ADR 0020.

The picture is a real gpt-4o-mini session sent through the proxy on 2026-10-21 by examples/context-map/live.mjs. Eleven requests, about 107k prompt tokens, under 0.01 USD. The agent reads this repo with tools. At turn 5 a skill tool is inserted near the front of the tool list, then a retry and an edited branch. OpenAI reported 0 cached tokens at that turn, so both signals fired. Later turns reused the previous prefix.

# Proxy already running, with AI_AUDIT_UPSTREAM_OPENAI_KEY and AI_AUDIT_CLIENT_KEY set.
AI_AUDIT_URL=http://127.0.0.1:8787 AI_AUDIT_CLIENT_KEY="aia-..." node examples/context-map/live.mjs
node examples/context-map/generate.mjs

generate.mjs builds a similar page from fake data and does not call a provider.

Operator console

The console is the place to watch the proxy and decide. It reads the log while the proxy writes it:

ai-audit console --log audit.jsonl --config gateway.json --reviewer ada \
  --anchors anchors.jsonl --metrics http://127.0.0.1:8787/metrics

It prints a URL with a token, for example http://127.0.0.1:8793/#token=…. Open that URL. The token is the only credential, so do not share it.

Page	What it shows
Overview	What needs attention now: decisions waiting for a review, held tool calls, refusals, a broken chain or a blocked budget. Suggestions from fixed rules, each with its evidence and the JSON to paste, for example a use case ready for a higher tier or a tool held again and again.
Decisions	The review queue. For each decision: the action the rule proposes, the result, the inputs, the program and its verdict. Run the rule again replays it in memory and lists each computed step. Approve, Override and Reject write the same record as approve.
Use cases	Each row of the use-case table with its calls, blocks, refusals, held tools, tokens and override rate.
Request flow	The state machine every request goes through: key, use case, route and body, model, inspection, decision, token budget, provider, tool gate, record. Each state shows how many requests reached it and where they stopped.
Requests	One row per request or refusal. A request opens with its path through the state machine and the context sent to the model: each block in prompt order, its kind, size and age, the first changed byte, and what made the context large.
Sessions and context	How the context of a conversation grew, turn by turn, with cache reads, writes and busts.
Keys and budgets	Each virtual key, its limits, its token budget and the tools its model called.
Integrity	Hash chain, segments, anchors and inspection profiles.
Live counters	The running proxy's /metrics, when --metrics is given.

The console binds only to a loopback address; reach it from another machine through an SSH tunnel. It refuses requests whose Host is not its own address. Without --reviewer it is read-only. A review note is at most 500 characters and is refused when it contains a secret or personal data. The console never shows prompt text, because the log does not have it, and never edits gateway.json. ADR 0054.

npm run example:workflow starts the same console over the synthetic log; open it from the dashboard with Operator console.

One request on its path through the state machine:

How the context of a five-turn session grew, by kind of block and by age. In turn 4 a tool definition changed, so everything after the first changed byte is new again and the provider cache cannot cover it:

Evidence workbench

Use the workbench to move from a Grafana signal to individual records without adding session IDs or content to /metrics:

ai-audit workbench --log audit.jsonl --output workbench.html \
  --from 2026-10-27T00:00:00Z --to 2026-10-28T00:00:00Z

The command verifies one hash-chained snapshot before it writes a new file. The HTML starts with expandable request investigations: a metadata fact sheet and a path through inspection, forwarding, provider result, tool observations and linked QDL decisions. Each stage labels observed, derived or unknown evidence. The rest of the page shows linked sessions, context sizes, usage, scan rule paths, Quinelings operations and reviews. It has local search and links between calls and sessions. Tables and investigations show the latest 500 calls; totals count the full interval. Narrow the interval to inspect older rows. Run the command again with a new output filename after new traffic.

The workbench omits retained output text, QDL source and inputs, reviewer names and notes. It cannot recover prompt text or a detected value because the proxy never stored them. The workbench is a private investigation file, not a shareable proof of origin. For one completed QDL decision, use receipt --format svg or --format json, inspect its exact contents, then give the recipient the artifact and verify-receipt. ADR 0037, ADR 0042.

Rotation and retention

The log path names a series. rotate, or serve when the file passes --rotate-bytes, renames audit.jsonl to audit.<UTC time>.jsonl under the append lock and starts a new audit.jsonl whose first record, kind: "segment", holds the closed file's name, record count and last hash. A name is never earlier than the segment before it, so a clock that steps back cannot reorder the series. The new file appears with its first record already in it. verify, report, stats, context, workbench, decide, replay and receipt read every segment in order. verify prints the file of a break, for example BROKEN at audit.20261029T110359005Z.jsonl line 3: hash mismatch. A segment that was changed, shortened or deleted breaks the series. Anchors keep working across rotations.

ai-audit rotate --log audit.jsonl
ai-audit verify --log audit.jsonl
ai-audit prune --log audit.jsonl --keep-days 184

prune deletes nothing by default and never touches the current file. It refuses a series that does not verify. Before it deletes the oldest segments whose last record is older than N days, it appends a prune record with each segment's name, count, head, first record hash and time range, so verify and verify --anchors accept that gap and no other. An anchor names the file it read by its first record hash, so an anchor of a deleted segment counts as pruned only when a prune record lists that segment. The AI Act asks for at least six months of logs from deployers of high-risk systems; 184 days covers any six months. Take a portable receipt of any decision that must outlive the window first. Backup tools and log shippers must follow the file name, not the open file. ADR 0049, ADR 0056.

Metrics

GET /metrics is OpenMetrics. Prometheus and VictoriaMetrics can scrape it. The bind address is 127.0.0.1 unless you change --host. The path has no authentication. If you expose the port, block /metrics in front of the proxy.

ai_audit_requests_total{route,decision,status}, ai_audit_request_duration_seconds from 5 ms to 300 s, ai_audit_log_unavailable_total, ai_audit_log_write_failures_total, ai_audit_refusals_suppressed_total, ai_audit_injection_signals_total{rule}, ai_audit_action_gate_total{mode,result}, ai_audit_quineling_events_total{where,op}, ai_audit_upstream_responses_total{provider,class}, ai_audit_upstream_errors_total{provider,reason}, ai_audit_unknown_usage_total{provider,field}, ai_audit_attachments_inspected_total, ai_audit_sandbox_unavailable_total, ai_audit_session_index_rebuild_failures_total, ai_audit_inspections_offloaded_total{result}, gauge ai_audit_sandbox_disabled, gauge ai_audit_attachment_cache_hits, gauge ai_audit_attachment_cache_misses.

Unknown paths are route="other". With a useCases table, ai_audit_requests_total and ai_audit_action_gate_total also have use_case: a row name, or none for a request that names no row. Labels are taken from fixed sets. A ready-made VictoriaMetrics, Grafana and ntfy stack is in examples/monitoring. ADR 0019.

On a loopback mock, Node 22, three runs, npm run bench:overhead measured about 0.6 to 0.7 ms p50 and about 1.1 ms p95 at concurrency 1, redaction included. At concurrency 16 one process levels off near 2000 requests per second. docs/performance/2026-10-20.

scrub

Export a conversation that is no longer active. Write a second file. Leave the export alone.

ai-audit scrub --input history.jsonl --output history.redacted.jsonl
ai-audit scrub --input conversation.json --output conversation.redacted.json > scrub-receipt.json

.json is one JSON document. Any other extension is JSONL. --format json or --format jsonl overrides that. JSONL is read one record at a time. The default limit is 32 MiB per record, and the same limit applies to a whole JSON document. --max-record-bytes N changes it.

The scanner walks prompts, assistant output, tool arguments, tool results, titles and other JSON strings, including a credential file pasted into a string. The input is opened read-only. Sanitized text is scanned again. If the source changes during the run, nothing is published. The output is written with mode 0600 and is never overwritten. Malformed JSON, bad UTF-8, duplicate keys, a secret in an object key, or a chained audit record fails with a static code and no partial file.

The receipt on stdout has the profile id, record and change counts, per-rule counts, and the digest of the sanitized file.

Scrub does not read data URIs or Anthropic base64 image payloads, and it does not open attachments. The receipt says so. It also does not delete the copy still sitting at the provider, in backups, or in the application you exported from. Import the sanitized file through that application's own path. ADR 0015.

check-quotes

The proxy returns the model's answer unchanged, so a misread number still reaches your code. Ask the model for each value together with a quote copied from the text it saw, then check both.

ai-audit check-quotes --source email.txt --answer answer.json

email.txt, the text the model actually saw:

Hello, please send an offer for 10 pieces of product A-200, premium version, delivery by 15 Nov.
Jan Nowak, jan@firma.pl, +48 600 100 200

answer.json, after the proxy had already masked the email:

{
  "quantity": { "value": 1, "quote": "10 pieces" },
  "product": { "value": "A-200", "quote": "product A-200" },
  "version": { "value": "premium", "quote": "premium version" },
  "contact": { "value": "Jan Nowak", "quote": "Jan Nowak, [REDACTED:pii-email]" },
  "material": { "value": "steel", "quote": null }
}

The command prints the object below and exits 1. Exit 0 means no issues. Exit 2 means a bad argument or an unreadable file. Nothing is written to the audit log.

{"ok":false,"checked":5,"issues":[{"field":"quantity","code":"value-not-in-quote"},{"field":"material","code":"quote-missing"}]}

Code	Meaning
quote-missing	No quote, or a quote made only of masks. Ask the model for "quote": null when the text does not state the value.
quote-not-in-source	That quote is not in the file.
value-not-in-quote	The number or the text is not inside its own quote.

Case, spacing and Unicode forms are ignored. [REDACTED:rule] in a quote stands for whatever was masked. A number matches an integer, a decimal such as 2,5, or a grouped number such as 10 000.

Pass --source the text the model saw, masks included. A wrong value with a genuine quote of some other sentence is not caught. A right value written as a word, a reformatted date, or a translation is reported. A piece of a decimal counts as found, so 10 is found inside 10,5.

From JavaScript, after npm run build:

import { checkQuotes } from "./dist/src/quotes.js";
console.log(checkQuotes(source, answer));

ADR 0031.

decide, approve, replay

The model proposes. A fixed policy decides. A person approves. replay runs the decision again from the log.

ai-audit decide \
  --policy examples/quineling-quote/policy.json \
  --inputs examples/quineling-quote/inputs.json \
  --system-id shop \
  --use-case quote \
  --llm-call "$REQUEST_ID" \
  --log audit.jsonl

ai-audit approve --decision "$DECISION_ID" --reviewer ada --verdict approved --note "within the floor"
ai-audit replay --decision "$DECISION_ID" --anchors anchors.jsonl
ai-audit receipt --decision "$DECISION_ID" --output receipt.html
ai-audit policies --system-id shop --use-case quote

--llm-call must already be an llm-call id in the same log. The response header can arrive before that row is appended, and decide in that window returns audit_unknown_llm_call.

A policy file is either a QDL intent, "format": "qdl-intent", or {"source": "<program source>"}. Inputs are a JSON object. Both are scanned before compile. Any finding refuses the decision and does not store the value.

Act on a decision after its verdict

A rule only proposes its actions; your application runs them. When the rule sends a decision to a person, the application waits, then records what it did:

ai-audit await --decision "$DECISION_ID" --timeout-s 1800   # exit 0 approved, 1 rejected or overridden, 3 timeout
ai-audit effect --decision "$DECISION_ID" --action send-quote-to-customer --status done --review "$REVIEW_ID"

await returns the latest review at once when there is one. overridden means the person takes the case over, so the application does not run the rule's action. effect stores the action name, done, skipped or failed, the review it followed and an optional code, never a payload or an amount. An action is authorized when the rule proposed it and no person rejected or overrode it, or when a person approved the decision before the action. A done action without that is recorded anyway, and the console lists it under Needs attention. From JavaScript, use awaitVerdict and log.effect. The console's Notify me shows a browser notification when a decision starts to wait; --notify-url URL also posts decision_waiting and, after --overdue-minutes (default 240), decision_overdue with the decision id, use case and proposed action names. ADR 0055.

replay exits 0 when the output matches and 1 when it does not. A refusal stored without its inputs cannot be rebuilt. That code is audit_qdl_not_replayable. node:vm replay of a plain JavaScript step is not a sandbox. Run only code you trust there.

npm run example:lab loads a fake session through the proxy and a mock provider, then compares two policies on it. The walkthrough, including what Quinelings will and will not do, is docs/decisions.md.

npm run example:evaluate checks a QDL machining quote policy on labeled cases. npm run example:evaluate:openai -- --output /private/tmp/quote-evaluation.json also evaluates the same facts with OpenAI Decisions, recording misses, false alerts, repeat changes, latency and usage. The evaluator runs offline from proxy traffic; see the evaluation guide.

npm run example:wycena -- --console prices five toolroom parts with a 64-node QDL rule and fixed company rates, then opens the operator console so each quote waits for a review. See the toolroom quote rule. examples/live-mail runs the same rule on four synthetic customer mails with the real proxy, a real model, the console in a browser and the CLI, each as its own process.

Configuration

gateway.json is JSON. Comments in the file make JSON.parse fail. This is a working file:

{
  "pageRedaction": true,
  "adminKeySha256": "paste-sha256-only-if-you-are-not-using-AI_AUDIT_ADMIN_KEY",
  "keys": [
    {
      "name": "agent",
      "sha256": "paste-the-sha256-keygen-printed",
      "models": ["*"],
      "providers": ["openai", "anthropic"],
      "logContent": false
    }
  ],
  "aliases": {
    "fast": { "model": "gpt-4o-mini" }
  }
}

Field	Meaning
keys[].name	Label stored on each record. Not a secret.
keys[].sha256	SHA-256 of the client key, as printed by keygen.
keys[].models	Model ids this key may request. "*" allows any. The check uses the name the client sent and the name after an alias.
keys[].providers	openai, anthropic, or both. Omit the field to allow both.
keys[].logContent	true stores redacted output text and redacted tool arguments. Default is hashes only.
keys[].tools	Optional mode and allow list of tool names, or prefixes ending in *, that this key's model may call. See Tool-call gate.
keys[].useCases	Optional list of useCases rows this key may name. See Use cases.
keys[].tokenBudget	Optional UTC period: day|month, nonnegative inputTokens / outputTokens, and positive per-request maxOutputTokens, each at most 2^48-1 (281,474,976,710,655).
actionGate	Optional tool-call gate: per tool name, mode, sensitive arguments paths and allow destinations. See Tool-call gate.
useCases	Optional table from a use-case name to its tier and optional models and tools. See Use cases.
budgetDirectory	Persistent local accounting directory shared by proxy processes. Default is <log>.budgets.
aliases	Map from the client's model string to { "model", "upstream" }. upstream is optional. The API format stays the same.
pageRedaction	false disables PDF rasterizing. Default is on. --no-page-redaction also disables it.
adminKeySha256	SHA-256 of the admin key. Ignored when AI_AUDIT_ADMIN_KEY is set, because the proxy hashes that variable itself.

allow.json is a JSON array of lowercase SHA-256 hex strings. Each string is the hash of one exact secret you have decided to let through. The file is not copied into the log.

["0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"]

Replace the placeholder. Each entry is the lowercase hex SHA-256 of one exact value. The digest of an empty string, or a hash that includes a trailing newline, does not match the value in the prompt.

Variable	Who reads it	Effect
AI_AUDIT_UPSTREAM_OPENAI_KEY	serve	Sent as Authorization: Bearer after a virtual key checks out.
AI_AUDIT_UPSTREAM_ANTHROPIC_KEY	serve	Sent as x-api-key on Anthropic routes.
AI_AUDIT_CLIENT_KEY	serve	Adds a virtual key named default, next to any keys in the config.
AI_AUDIT_CLIENT_LOG_CONTENT	serve	Set to 1 to turn on logContent for that default key.
AI_AUDIT_ADMIN_KEY	serve	Unlocks GET /audit/stats and GET /audit/budgets. The hash of this value wins over adminKeySha256.
AI_AUDIT_UNSAFE_NO_SANDBOX	serve	Set to 1 for the same effect as --unsafe-no-sandbox.
AI_AUDIT_URL, AI_AUDIT_CLIENT_KEY	examples/context-map/live.mjs	Where the example sends its session. The proxy does not read AI_AUDIT_URL.

If virtual keys are on and the upstream variable for that provider is empty, the proxy strips the client's credential and calls upstream with no key.

Architecture

Claude Code, Codex, SDK, curl
        |  Authorization: Bearer aia-...
        v
+--------------------------------------------------+
| serve                                            |
|  1. virtual key, model, provider                |
|  2. JSON limits, log can be appended            |
|  3. alias, then scan every string               |
|  4. attachments in bwrap: OCR, PDF, image       |
|  5. redact, block, or keep the original bytes   |
+--------------------------------------------------+
        |                         |
        | one llm-call            | HTTPS, provider key
        v                         v
  audit.jsonl              api.openai.com
  <log>.session-key        api.anthropic.com
  <log>.lock
        |
        |  ai-audit anchor, stored elsewhere
        v
  anchors.jsonl

Later, not on this path:
  decide -> approve -> replay
  scrub of an exported history
  check-quotes of one answer

One request with the log, the disk, the client and the provider failing at any step, drawn with /statechart from spec/RequestLifecycle.tla (docs/verification/request-lifecycle.cfg, 148 model states). It starts after the key, route and body checks; requests stopped there are refused as in the chart above, and GET /healthz, /readyz, /metrics, /audit/stats with the admin key and GET /v1/models are answered without a record.

---
title: "One request through the proxy, with the log, the disk, the client and the provider failing at any step"
---
stateDiagram-v2
  state "Legend" as legend {
    lg0: decide
    lg1: provider
    lg2: end
    lg3: Breaks NoForwardWithoutLog
    lg4: Path to the NoForwardWithoutLog bug
  }
  class lg0 blue
  class lg1 yellow
  class lg2 teal
  class lg3 breaks
  class lg4 bugPath
  s0: Request received
  s1: Stopped, failure counted
  s2: Inspected and decided
  s3: Stopped and recorded
  s4: Refused, refusal recorded
  s5: Forwarded to the provider
  s6: Answer streaming to the client
  s7: Answer delivered, record not written yet
  s8: Delivered, record lost and counted
  s9: Delivered and recorded
  [*] --> s0
  s0 --> s1: the log is broken: 503
  s0 --> s2: the log is writable
  s2 --> s3: blocked or rejected, recorded
  s2 --> s4: an internal error
  s2 --> s5: the log is still writable, the proxy forwards
  s5 --> s3: no answer from the provider, recorded
  s5 --> s6: the provider answers
  s2 --> s1: the log broke during inspection: 503 or blocked or rejected, but the record fails or an internal error
  s5 --> s1: no answer from the provider, the record fails
  s6 --> s1: an internal error
  s6 --> s7: the answer is delivered
  s7 --> s8: an internal error or the record fails
  s7 --> s9: the record is written
  classDef breaks fill:#ffecee,stroke:#e72e3b,stroke-width:2.5px,color:#cd2834
  class s0 blue
  class s0 bugPath
  class s1 teal
  class s2 blue
  class s2 bugPath
  class s3 teal
  class s4 teal
  class s5 yellow
  class s5 bugPath
  class s6 yellow
  class s6 bugPath
  class s7 yellow
  class s7 bugPath
  class s8 teal
  class s8 breaks
  class s8 bugPath
  class s9 teal

Loading

Delivered and recorded appends exactly one hash-chained llm-call. Refused, refusal recorded forwards nothing and appends one content-free refusal record; a broken log is refused with 503 and no record. Delivered, record lost and counted is the window described in ADR 0016, red in the chart. The client already has the body, so the failure is counted in ai_audit_log_write_failures_total and printed on stderr as audit_log_write_failed. The reason is an error code such as ENOSPC, or a class such as TypeError, never the message. The append lock and the anchor check are drawn in docs/verification. TLA+ models live in spec/.

Attachment workers run under bubblewrap with no network, no capabilities, and an empty environment. Read-only mounts are /usr, the Node install, and the worker script. A worker cannot read the log, the session key, the config, or your home directory. Limits, from the preflight: worker RLIMIT_DATA 4 GiB, CPU 90 s, files 64 MiB, 128 open files. Each native tool also gets RLIMIT_AS 1.5 GiB. Wall clock is 120 s. Output caps are 16 MiB for the worker and for each tool. Crossing one of those is 422.

Masking, context blocks and usage totals are pure functions with proved laws. They are written in Bend 2 and shipped as generated JavaScript. ADR 0029.

Troubleshooting

npm install -g ai-audit scans a directory

That name on the npm registry is a different tool. Clone this repository and run node dist/src/cli.js, or npm link after npm run build.

401 audit_client_unauthorized

Virtual keys are configured, and the bearer token is not one of them. The log has a refusal record with the time, route and source address, never the token; ai-audit report counts refusals by code. Use the aia- string keygen printed. The config must contain the sha256 from that same printout, not the key itself.

A 401 from the provider, after your proxy returned 200 from /healthz, usually means AI_AUDIT_UPSTREAM_OPENAI_KEY or AI_AUDIT_UPSTREAM_ANTHROPIC_KEY is unset. With virtual keys on, the client credential is removed and nothing is put in its place.

GET /audit/stats returns 401 audit_admin_unauthorized when x-admin-key does not match AI_AUDIT_ADMIN_KEY.

400 audit_use_case_required or 403 audit_use_case_not_allowed

gateway.json has a useCases table. The client must send x-audit-use-case with a row name that its key may use, or use a key whose useCases list has one row. The name must match exactly, in lower case. 403 audit_use_case_off means the row has tier: "off". Use cases.

503 audit_log_unavailable, or verify prints BROKEN

The last line was torn, the disk is full, or the process cannot create <log>.lock. The proxy refuses new inspected calls until this is fixed. GET /healthz can still say ok. GET /readyz does not check the log.

ai-audit verify --log audit.jsonl
df -h .
ls -l audit.jsonl audit.jsonl.lock

Repair a torn last line only after you know which bytes were cut. Then verify again. If you keep anchors, check those too:

ai-audit verify --log audit.jsonl --anchors /elsewhere/anchors.jsonl

A failure after the client already received the response is a stderr line audit_log_write_failed. That request will not grow a second log row.

422 on a request with a file

/readyz tells you which prerequisite is missing.

/readyz or error code	Fix
503 audit_tools_missing	Install tesseract-ocr, poppler-utils, python3-pil, and util-linux for prlimit.
503 or 422 audit_sandbox_unavailable	bubblewrap has to work. On Linux, bwrap --version. In Docker, add the flags in Docker. macOS has no prlimit and no usable bwrap here. Text-only calls still work.
422 audit_unsupported_attachment	Only PNG, JPEG, WebP, GIF and PDF are inspected.
422 audit_uninspectable_reference	A file id or a remote URL cannot be fetched and scanned. Enforcing modes refuse the call. advisory allows it.
422 audit_inspection_failed	The file is corrupt, a limit tripped, or a tool failed. The call is not forwarded.
422 audit_inspection_timeout	The request text took longer than --inspect-timeout-ms to scan. A text of many MiB built from one repeated pattern does this. Raise the deadline only if such text is expected.
503 audit_inspection_busy	All scan workers are busy and 64 more requests wait. The client can retry.

--unsafe-no-sandbox skips the sandbox check. Do not use it on a machine that receives untrusted files.

409 audit_findings_blocked or audit_unredactable_finding

no-go blocks every finding. redact blocks when the hit cannot be replaced in place. The findings array is rule ids.

To let one known value through, hash those exact bytes. Add no newline.

printf '%s' 'the-exact-secret' | node --input-type=module -e 'import {createHash} from "node:crypto"; const chunks = []; for await (const c of process.stdin) chunks.push(c); console.log(createHash("sha256").update(Buffer.concat(chunks)).digest("hex"))'

Put that lowercase hex string in allow.json and restart with --allow-hashes allow.json. --mode no-go refuses the call. --mode advisory logs the finding and forwards the original body.

Token totals stay low, or unknownUsage climbs

OpenAI's streaming chat API omits usage unless the client asks for it. The proxy will not insert the flag.

"stream": true,
"stream_options": { "include_usage": true }

Rows with tokensComplete: false are a lower bound. Do not add the missing calls in as zero.

decide prints audit_qdl_unavailable

The optional package did not install. From the repo:

npm ci
node -e 'import("@quinelings/agent-sdk/v1").then(() => console.log("ok"))'

The proxy's serve command does not need this import.

Limitations

• No person-name detection, no street addresses, no US Social Security numbers. Phone coverage is international numbers and one grouped digit pattern, not every national format.
• No translation between OpenAI and Anthropic request bodies.
• Attachment inspection needs Linux, bubblewrap, prlimit, Tesseract, poppler and Pillow. Without them, calls that carry a file fail closed.
• Inspected types are PNG, JPEG, WebP, GIF and PDF. Other files, file ids and remote URLs are refused in redact and no-go.
• The attachment cache is memory only. It dies with the process.
• CLI readers hold every record of the log series in memory. A long series needs that much memory; prune bounds it.
• Requests refused with 401, 403, 400, 404 or 413, and requests stopped by an internal error before forwarding (500, with the error class or code as reason), are refusal records without content, at most 60 per minute per process; the rest of that minute is one summary record with a count (ADR 0047). count_tokens is scanned and not logged.
• scrub rewrites an export you already have. It does not reach the provider, backups, or data URIs.
• Prompt-injection rules never block, and they are easy to walk around. The tool-call gate blocks only listed tools, keys with a tools list and destinations it can parse.
• check-quotes misses a wrong value backed by a real quote of a different sentence. It flags numbers written as words, translated text, and reformatted dates.
• A rasterized PDF has no text layer. Answers that depend on the original text get worse, and the token count goes up.
• The AI Act discussion in DESIGN.md is unverified. It is not legal advice.
• replay of a trusted JavaScript step uses node:vm. That is not a sandbox.

FAQ

Why a proxy?

The secret often shows up in a tool result at request time. A pre-commit hook never sees that payload. The SDK already has a base URL, and the proxy takes that slot.

Is the log a compliance certificate?

No. It is a hash-chained record of what this process inspected and what it forwarded. DESIGN.md sketches an Article 12 style mapping. Nobody has certified that mapping, and this README is not legal advice.

Does the provider see my provider key in the prompt?

The provider key is added as the upstream credential after the scan, from the proxy environment. It is not taken from the logged body. A provider key that a user pasted into the prompt is a finding, same as any other key of that shape.

What leaves the machine?

The forwarded request, original or redacted, goes to the upstream URL. Default is https://api.openai.com or https://api.anthropic.com. Metrics, the audit log and the session key stay local. /metrics publishes counters to whoever can open the port.

Do I have to run Quinelings?

No. serve, verify, scrub, check-quotes and redaction work without it. Install it when a model writes the policy, or when the policy source has to survive a process restart inside the log. A short trusted function can be recorded with the built-in step log instead. See docs/decisions.md.

Why did verify succeed and an anchor check fail?

verify alone checks that each line's hash matches the previous line. An attacker who replaces the whole file with another valid chain still passes. --anchors compares the chain to a digest you stored elsewhere.

Why is /readyz failing while chat still works?

/readyz is 503 when attachment tools or bubblewrap are missing, even if every current request is plain text. Plain text does not need those tools. The first request that includes a file will get 422 until you install them, or until you set --unsafe-no-sandbox.

Can I turn the proxy off for one call?

Point that client back at the provider. There is no per-request bypass header. advisory still logs, and it still refuses a body the parser will not accept.

Development

npm run build is tsc. Tests, the demo and the benchmarks run against dist/. npm run build:fast uses the pinned tsrs compiler and is about 15 times faster for local edits. It needs --skipLibCheck because tsrs reports TS5115 on circular types in Zod's declaration files. tsrs is a dev dependency.

npm test builds, then runs node --test on dist/test/*.test.js. The seeded generators in test/fuzz.test.ts are part of that run. npm run fuzz runs them 50 times longer. FUZZ_SCALE and FUZZ_SEED change that. npm run fuzz:coverage runs the Jazzer.js targets in fuzz/ for FUZZ_SECONDS each. The default is 300. Jazzer.js has native code, so it lives in fuzz/package.json and is not part of the main install. It cannot see inside regular expressions. Slow-regex inputs are caught by timing a repeated pattern. A finding saved under fuzz/artifacts/ should become a regression test.

Measured tables in the docs are generated, not typed. npm run results checks them against the committed data and npm run results -- --write rewrites them; npm test fails when one is out of date. ADR 0052.

CI runs npm test on Linux with Node 22, 24 and 26, with the attachment tools and bubblewrap installed, so those tests are not skipped. It also runs npm run verify:bend and spec/run.sh on every push to main and every pull request. Each night it runs npm run fuzz and 600 seconds of Jazzer.js per target, and it keeps the corpus. ADR 0030.

sh bend/install.sh   # pinned Bend 2.0.38, sha256 checked
npm run verify:bend
npm run verify:formal

About contributions

Issues and pull requests are welcome. Report a vulnerability privately, as SECURITY.md describes. There is no separate contributing guide. Run npm test before you push. Do not commit secrets, real API keys, or real audit logs. live/ and *.jsonl from real traffic stay out of git. Tests use fake keys.

The license is MIT, copyright Adrian Góra.

License

MIT. Detection rules adapted from Gitleaks. See THIRD_PARTY_NOTICES.md. The vendored Quinelings SDK is Apache-2.0. See vendor/QUINELINGS-LICENSE.