Run a workspace's stored guardrail definitions on every request

[dpoulopoulos]

The gap

Three stacked PRs built a guardrail Otari runs itself, and stopped one step short of using it.

• feat(guardrails): store guardrail definitions in the database #1211 added the guardrail_credentials table and its CRUD API.
• feat(guardrails): build every stored guardrail at startup and run it in this process #1244 builds every stored row into a live any_guardrail object at startup and after a write, and added POST /{name}/test.
• feat(dashboard): add the page that defines a guardrail otari runs itself #1256 added the dashboard page that creates those rows.

An operator can define a guardrail, store its credential, switch it on and test it. None of that changes what happens to a request. docs/guardrails.md says so outright:

Nothing on the request path reads these rows yet, so storing a definition still does not change how a request behaves.

The proof in code: GuardrailRunner.check() (src/gateway/services/guardrail_runner.py) has zero callers. Everything on the request path still goes over HTTP to the sidecar container at guardrails_url.

What this adds

A definition that is stored, enabled and scoped to a workspace checks the input of every inference request from that workspace, before the provider is called. The caller sends nothing; there is no guardrails field to fill in.

The design splits one question into two, and keeping them apart is what keeps the change small:

Question	Answered by
Which guardrails apply to this request?	the database, scoped on the request's workspace
How is one run?	GuardrailRunner, which already built them all at startup

So GuardrailRunner needs no change at all. knows() and check() already exist and already do the right thing. Nothing is built while a request waits.

Schema

Three columns on guardrail_credentials, and one scope table.

• mode: block or monitor. What to do when the guardrail flags the input.
• on_unavailable: block or allow. What Otari does when no verdict came back at all.
• applies_to_all_workspaces: a flag, with guardrail_credential_workspaces holding the explicit list otherwise. Both copy organization_guardrails and organization_guardrail_workspaces column for column.

A guardrail is unavailable when the vendor API failed or timed out, or the answer was malformed. An explicit inconclusive verdict (valid: null) is not unavailable and never blocks.

The permissive value is spelled allow rather than the legacy monitor used by the request-body and organization fields, because a guardrail that never answered monitors nothing, and the decision is Otari's rather than the guardrail's. One mapping turns allow into monitor where a GuardrailConfig is built; the published request-body contract is not renamed.

Request path

The workspace's definitions become one more mandate layer on the existing merge, applied after the caller, the organization and the routing policy. That reuses the union by profile with stricter-wins, so a caller who also names the profile is deduped rather than checked twice and cannot weaken it. Every refusal shape is the existing one: 403 guardrail_violation, 502 fail closed, and the X-Otari-Guardrails header for monitor mode.

run_input_guardrails gains one branch: run in process when the runner knows the profile, else POST to the sidecar as today. A stored definition beats a sidecar profile of the same name.

Known limits, deliberate

• A definition that failed to build is skipped, so the request goes through unchecked. To stop that being silent the read endpoint reports whether the runner holds the profile, and the dashboard row says so.
• Enabled rows are capped at ten, because each one is another sequential vendor call in front of a request. The checks run sequentially; making them concurrent touches the sidecar path too and belongs in its own change.
• Pass-through routes (embeddings, images, audio) do not go through the shared pre-dispatch seam and stay unchecked.

Stack

Last PR in the #1211, #1244, #1256 stack. Based on feat-guardrail-dashboard.

[dpoulopoulos]

Closing. Superseded by an organization-scoped design.

The gap this issue describes is real and still the goal: a stored, enabled, workspace-scoped definition should check every request from that workspace, with no guardrails field in the body. What changes is where the definition lives. Rather than adding mode, on_unavailable and applies_to_all_workspaces to guardrail_credentials plus a new scope table, the definition moves onto organization_guardrails, which already carries all four and is already resolved per request.

So the schema half of this issue is no longer needed, and the request-path half gets smaller: run_input_guardrails takes one extra argument and grows one branch.

Two things found while planning it that this issue did not cover:

• Guardrail.validate() is synchronous and there is no async variant, so it has to run in a bounded executor of its own. A thread cannot be cancelled, so a hung vendor call outlives the request that started it, and the process-wide default executor is shared with file extraction.
• The in-process status has to survive the layer merge as its own field. After _overlay_mandate, an in-process entry's url is None, which is indistinguishable from a remote entry falling back to the deployment's guardrails_url.

The "known limits, deliberate" section stands as written, including the sequential checks and the pass-through routes staying unchecked.