feat(tools): run provider-native code execution on the sandbox when the model has none, with Files API parity

[daavoo]

Description

A request that declares a provider's own code-execution tool (Anthropic's code_execution_20250825, OpenAI's code_interpreter, or the bare code_execution) is today always forwarded to the provider. Only the Otari-specific otari_code_execution type reaches the gateway's sandbox. So a client built against Anthropic's sandbox breaks the moment its model is swapped for one without a native sandbox, and the files it attaches break with it.

What changes. The gateway now decides who runs such a declaration, in three settings:

• auto (the new default): the provider when it runs that tool natively for the dispatched model and wire format; otherwise the gateway's sandbox, answering in the caller's own vocabulary (server_tool_use and code_execution_tool_result blocks on Messages, a code_interpreter_call item on Responses).
• otari: always the gateway's sandbox.
• provider: always forwarded, exactly as before.

The deployment sets the default (code_execution_executor, also on the dashboard's Tools page), a workspace policy may pin a value, and the X-Otari-Code-Execution header picks per request where the workspace has not pinned. A deployment without a sandbox is untouched: provider declarations are forwarded and no policy is read. otari_code_execution keeps working unchanged.

Behavior change for a deployment that already has sandbox_url set. Under the new default, a provider-named declaration (code_interpreter, code_execution_<date>, code_execution) against a model whose provider does not run it natively used to be forwarded and refused upstream; it now runs on the deployment's sandbox and is billed as a sandbox tool call, and GET /api/v1/tools lists those keywords under otari_code_execution.accepted_types. Set code_execution_executor: provider to keep the old behavior.

The workspace policy is read once, in the request preamble, so the same decision says whether a referenced upload is staged for the sandbox and who runs the code; a pin can no longer leave a request without its attachments. A file a provider's own sandbox produced is recorded on the streaming path as well as the non-streaming one, through the pipeline's settlement rather than in each route, keyed to the configured provider instance that served the request.

Files follow the same rule. This absorbs and supersedes #1364 (Files API parity) and wires it to the executor:

• Both SDKs' Files APIs are served on the same five routes (Anthropic FileMetadata for a caller sending anthropic-version, the OpenAI file object otherwise). Listings are cursor-paged, an fsspec storage backend joins local and s3, and a background sweep reclaims expired and deleted files.
• Uploads a request references are seeded into the sandbox session whenever the code runs here, whichever declaration brought it. When the provider keeps the declaration, an Anthropic container_upload is read as a document, as it is today.
• Files the code produces are streamed into the store as code_execution_output files owned by the same user and workspace, at most files_output_max_files and files_output_max_bytes per call (20 files, 64 MB). An Anthropic-native caller gets their file_id in the result block, where its SDK looks.

Follow-ups, not in this PR: a CodeExecutionPort with an e2b adapter, cross-request container ids, files in hybrid mode, cfile_ ids on Responses, a platform-side executor pin in hybrid mode, a public any-llm capability flag to replace the private-attribute check that decides which betas a bridged provider can take (mozilla-ai/any-llm#1418), and an any-llm files API to replace the hand-rolled provider download in provider_files.py (mozilla-ai/any-llm#1419). The fsspec backend ships as the optional otari[fsspec] extra, like otari[s3].

How to test it locally

1. docker compose --profile code-exec up, then send an Anthropic Messages request with tools: [{"type": "code_execution_20250825", "name": "code_execution"}] against a non-Anthropic model. The code runs on the sandbox and the response carries server_tool_use and code_execution_tool_result blocks with otari_srvtoolu_ ids.
2. Send the same request against an Anthropic model: it is forwarded untouched. Add X-Otari-Code-Execution: otari and it runs here instead.
3. Upload a file with the OpenAI SDK and with the Anthropic SDK, both pointed at the gateway; each gets its own response shape, and GET /api/v1/files?limit=2 pages with has_more and last_id. Reference it from a request that runs on the sandbox: the file is in the session's working directory, and a file the run writes comes back as a file_id in the tool result.
4. Tools page: the Code execution group has a "Who runs provider code tools" select; the workspace card below it offers the same choice as a pin. GET /api/v1/tools lists the provider keywords under otari_code_execution.accepted_types unless the default is provider.

demo/code-exec/qa_executor.py walks all four against a running gateway.

Automated coverage: tests/unit/test_code_executor.py (the decision and when attachments are staged), tests/integration/test_code_execution_executor.py (request path, header, pin, folding), native block minting in tests/unit/test_mcp_loop_messages.py and tests/unit/test_mcp_loop_responses.py, echoed-block folding in tests/unit/test_messages_minted_block_stripping.py, the files tests (tests/integration/test_files_endpoint.py, tests/unit/test_sandbox_backend.py, tests/unit/test_content_normalizer.py, tests/unit/test_fsspec_file_store.py), plus the tool-settings, tools-discovery and dashboard tests. make lint, make typecheck, the unit and integration suites, the OSS smoke gate and the dashboard suite pass locally.

PR Type

• New Feature
• Bug Fix
• Refactor
• Documentation
• Infrastructure / CI

Relevant issues

Supersedes #1364 (and through it #976 and #977). Part of #786. Groundwork for the code-execution provider abstraction discussed alongside #896.

Checklist

• I understand the code I am submitting.
• I have added or updated tests that cover my change (tests/unit, tests/integration).
• I ran the Definition of Done checks locally (make lint, make typecheck, make test).
• Documentation was updated where necessary.
• If the API contract changed, I regenerated the OpenAPI spec (uv run python scripts/generate_openapi.py).
• If this changes a rule in ARCHITECTURE.md or scripts/check_architecture.py, the description names the rule and says why.

AI Usage

• No AI was used.
• AI was used for drafting/refactoring.
• This is fully AI-generated.

AI Model/Tool used: Claude Code (Claude Fable 5.1)

Any additional AI details you'd like to share: Design agreed with the maintainer in conversation. The executor half was implemented, tested and documented by the agent; the files half is the maintainer's #1364, merged in and adapted to the executor by the agent. Everything was verified locally.

• I am an AI Agent filling out this form (check box if true)

🤖 Generated with Claude Code

Summary

• Added configurable routing for provider-native code execution with auto, otari, and provider modes.
• Added sandbox file staging, output collection, size limits, and provider-compatible Messages and Responses results.
• Extended the Files API with OpenAI and Anthropic formats, cursor pagination, provider-held files, configurable storage, and automatic cleanup.
• Added dashboard controls, documentation, SDK demos, database migration, and broad test coverage.

Technical notes

• Provider-produced files can remain in the provider container and stream through Otari on demand.
• Deployments without a configured sandbox continue forwarding provider declarations.
• The migration adds executor policy, provider file metadata, nullable storage references, and listing and expiry indexes.

[coderabbitai]

Understand this PR’s impact

Explore downstream dependencies and potential security impact with Blast Radius.

View blast radius →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: mozilla-ai/otari/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 7930b9b3-e676-4fff-910c-911043b8b5dd

📥 Commits

Reviewing files that changed from the base of the PR and between 66a84d4 and a81ed09.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock, !**/uv.lock

📒 Files selected for processing (7)

• docs/files.md
• pyproject.toml
• src/gateway/api/routes/messages.py
• src/gateway/core/config.py
• src/gateway/services/file_store.py
• src/gateway/services/files/provider_files.py
• tests/unit/test_fsspec_file_store.py

🚧 Files skipped from review as they are similar to previous changes (5)

• docs/files.md
• src/gateway/services/file_store.py
• src/gateway/core/config.py
• src/gateway/services/files/provider_files.py
• src/gateway/api/routes/messages.py

Included review availability: Your plan provides up to 2 included reviews per hour; 0 remain after this review.

---

Walkthrough

This pull request adds configurable code-execution routing, sandbox file transfer, provider-held file support, cursor-paged file APIs, fsspec storage, retention sweeping, frontend controls, documentation, demos, and tests.

Changes

Code execution and files

Layer / File(s)	Summary
Executor contracts and configuration src/gateway/types/*, src/gateway/core/config.py, src/gateway/models/tools.py, alembic/versions/*	Adds the auto, otari, and provider executor vocabulary, deployment and workspace settings, provider-file metadata, output limits, fsspec settings, and the related migration.
Executor routing and sandbox admission src/gateway/api/routes/_tools.py, src/gateway/api/routes/_pipeline.py, src/gateway/api/routes/_normalize.py, src/gateway/api/routes/{chat,messages,responses}.py	Resolves executor precedence, validates request headers, detects provider-native declarations, stages inputs, and passes sandbox file bridges into tool execution.
Sandbox outputs and native responses src/gateway/services/sandbox_backend.py, src/gateway/services/mcp_loop_*.py, src/gateway/services/file_service.py	Seeds sandbox inputs, collects bounded outputs, records executions, persists generated files, and emits native Anthropic or Responses code-execution results.
File APIs and storage src/gateway/api/routes/files.py, src/gateway/services/file_store.py, src/gateway/services/files/*, src/gateway/main.py	Adds Anthropic response shapes, cursor pagination, provider-held file streaming, fsspec storage, and scheduled retention cleanup.
Validation and product surfaces tests/**/*, web/src/**/*, docs/**/*, demo/code-exec/*, config.example.yml	Adds coverage for routing, files, sandbox behavior, provider access, frontend settings, API schemas, documentation, and SDK demos.

Priority: ➖ Normal

Estimated code review effort: 5 (Critical) | ~90 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title accurately describes the main change and uses an acceptable scoped Conventional Commit form, but it is 109 characters and exceeds the approximately 70-character limit.	Shorten the title to approximately 70 characters or fewer while preserving the main change, for example: "feat(tools): route provider code execution to sandbox".
Docstring Coverage	⚠️ Warning	Docstring coverage is 36.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 501 functions across 55 files. (2 skipped…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description is complete and relevant. It includes the change summary, testing steps, PR type, related issues, checklist status, AI usage, and follow-up scope.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 36.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 501 functions across 55 files. (2 skipped: 2 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

✨ Simplify code

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}