Serve provider-native Files in standalone mode

[HareeshBahuleyan]

Desired behavior

Standalone mode serves the same provider-native Files API that #1185 adds for hybrid: upload, list, retrieve, download, and delete files at Anthropic or OpenAI through the official SDKs, and reference them from Anthropic Messages. Bytes stay at the provider; the local database holds ownership, quotas, and the cleanup queue.

Why

#1185 gates hybrid_files.router on is_hybrid_mode, so a standalone deployment with an Anthropic or OpenAI key cannot use container_upload or persistent provider files. It has to inline the file on every request through the local /api/v1/files extraction path instead.

The core is already mode-neutral: ProviderFileService, create_provider_files_router, the any-llm transport, the envelope adapters, and migration c3e5a7b9d1f4 all land in standalone too. What is missing is the wiring:

• Standalone authenticate / authenticate_gateway / authorize_attempt callbacks for create_provider_files_router, resolving the local API key to a FileScope.
• A local account resolver that mints ProviderAccountGeneration rows from provider_credentials and puts provider_account_generation_id on standalone attempts.
• An in-process sibling of PlatformFilesClient so hybrid_files.py can reach ProviderFileService without the HTTP hop, and an in-process cleanup executor in the standalone lifespan.

Notes

• Blocked by feat(files): support provider-neutral Files API support for hybrid mode #1185 merging.
• Hosted mode stays out: it does not mount the inference routers, and Hosted control plane serves the data-plane inference endpoints unbilled #822 keeps inference off the hosted data plane.

Ref: #984

[daavoo]

Notes from the Octonous side, as a standalone consumer. Octonous plans to start fully self-hosted and move to otari.ai later, so standalone is the mode it is judged on and hybrid is not a step it takes. None of the below argues against doing this work; it is about what the standalone half is for and what it still owes.

Two corrections to the framing

container_upload already works in standalone. content_normalizer.py stages an Anthropic container_upload block into Otari's own sandbox, so the block is honored today. What is gated is container_upload against the provider's native code execution, which is narrower than "a standalone deployment cannot use container_upload".

The container instability is fixed. #1433 merged: Otari's sandbox is held across requests and reported as a container id, an id it minted pins the executor so a model swap under auto no longer takes the sandbox away mid conversation, and a resumed workspace keeps its files. That was the one hard breakage Octonous reported, and it is closed.

What that leaves for standalone

One gap, and it is a cost gap rather than a capability gap. Without provider-native files, a referenced upload is resolved to inline bytes on every turn, so a large attachment is re-transmitted per request. Token cost and cacheability are much the same either way, so this is bandwidth and latency. Nobody is locked out.

So for standalone this reads as an optimization rather than an unblock. That changes how closely it needs to follow #1185, not whether it is worth doing.

The part that decides whether it is worth adopting

A provider copy has to refresh itself, or standalone gains nothing.

#1185 records provider_expires_at, floors its own clock to it, and once past it the binding stops being visible and the file answers 404. There is no re-upload path. That is correct for hybrid: the provider holds the bytes, so when the copy expires there is genuinely nothing to re-upload from.

It is the wrong behavior for standalone, by the direction set in #1488. Point 2 there says Otari's store is the source of truth and a provider copy is a short-lived cache. A cache that 404s instead of refilling is not a cache, it is a pointer with an expiry. Standalone does hold the bytes, so it can refill.

Concretely: a user returns to a conversation after two days. The Gemini copy is gone (its Files API deletes after 48 hours), the OpenAI code-interpreter container went 20 minutes after last use, the Anthropic file is still there. Octonous handles this today by filling provider copies lazily per provider. If standalone adopts #1185's behavior unchanged, Otari answers 404 for a file whose bytes it is holding, Octonous keeps its lazy-fill logic, and it has gained a second place for the same problem to live rather than one fewer.

With refresh on use, the caller sends one Otari file id and never learns that an expiry happened. That is what lets a consumer delete a subsystem: per-provider id columns, expiry detection, re-upload, and reaping on delete. The per-turn bandwidth saving is the smaller half of the value.

The model swap falls out of the same design. Keyed on (file id, provider, instance), switching provider mid conversation is a cache miss, so the copy is made on that turn. That is exactly why a consumer keeps one copy per provider today.

Worth naming the cost: the refresh happens inline, so the first turn back after an expiry pays an upload before the model is called. Octonous pays that today too, so it is not a regression, but it moves into Otari's latency budget and should be measured rather than assumed invisible.

Standalone may not need to wait for #1185

Because Otari holds the bytes, losing a provider copy is harmless: re-upload. That removes the reason for most of #1185's machinery, which exists because in hybrid a lost copy is unrecoverable. No two-phase commit over the upload, no compensation protocol, no cross-gateway cleanup leases, no cross-process upload admission. What standalone needs is closer to a mapping table from (file id, provider, instance) to a provider file id with the provider's expiry, upload on demand, refresh on miss, and a sweeper.

That is a different and much smaller design than the one in #1185, not a port of it. Worth deciding explicitly whether standalone is blocked by #1185 merging, or only by agreeing this shape.

Two things nobody owns yet

Gemini is an any-llm change, not an Otari one. any-llm 1.28.0 ships a files API, but gemini reports SUPPORTED_FILE_OPERATIONS = []; only Anthropic and OpenAI are covered. mozilla-ai/any-llm#1410 is open for exactly this. If the standalone work keys off SUPPORTED_FILE_OPERATIONS rather than naming providers, Gemini needs no Otari change at all and starts working when that ships. Octonous holds a gemini_file_id alongside the other two, and it is a URI rather than a file- style id, so this is the difference between one file system and two.

"Otari's store is the source of truth" quietly assumes a durable backend. files_backend defaults to local, a directory at ./otari-files, and files_retention_hours defaults to None so nothing expires. On a PaaS with ephemeral disk or more than one replica, the default loses the bytes on restart and hides them from other replicas, which makes the source-of-truth claim false exactly where provider copies matter most. One setting fixes it (files_backend: s3), but nothing says so today, and a deployment that turns provider files on without it gets a cache whose source of truth can vanish.

Sequencing

From a standalone consumer's point of view, #1475 looks like the most valuable unblocked work rather than this issue. It is unassigned, independent of both #1185 and this one, and it fixes a live bug: Octonous downloads code-execution output straight from the provider, and OpenAI discards the container 20 minutes after last use, so an older chart is simply gone. It is also lower risk than it sounds, because since #1433 Otari's own sandbox already streams produced files into Otari's store through SandboxFileBridge. #1475 brings the provider-native path to parity with a path that already exists and is tested.

After #1475, this issue with refresh written in, any-llm#1410, and #1487 for deletion, a consumer keeps exactly one thing: the association between a conversation and its attachments. Everything else is Otari's. That is the story worth putting in front of other users.

_{Written by Claude Code from the Octonous standalone perspective; facts checked against main at the time of writing.}