feat(serving): add OrcaRouter as a named LLM provider

[nissrin2020ali-ux]

Adds APIOrcaRouterServing, so DataFlow users can drive their whole data-preparation stack (generation, evaluation, filtering) through the OrcaRouter gateway the same way they already point an LLM serving at OpenAI — but as a first-class named provider instead of an anonymous custom base URL.

OrcaRouter is an OpenAI-compatible AI gateway built for both models and agents. Like OpenRouter, it exposes a provider/model namespace across many models behind a single endpoint — but it also combines adaptive routing, automatic failover, zero-markup inference, observability, guardrails, and agent-tool governance on the same endpoint. Adding orcarouter as a named backend in dataflow/serving means this project's LLMServingABC-based operators and pre-configured LLM backends can use that stack directly.

The implementation mirrors the existing APILLMServing_request path: APIOrcaRouterServing subclasses it and only wires in OrcaRouter defaults (base URL https://api.orcarouter.ai/v1/chat/completions, ORCAROUTER_API_KEY, and the orcarouter/auto model namespace), reusing the parent's request, retry, timeout, and response-formatting logic unchanged. Usage is the same shape as the README's operator example:

from dataflow.serving import APIOrcaRouterServing
llm_serving = APIOrcaRouterServing(model_name="orcarouter/auto")

It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Verification:

• Unit tests added under test/cpu_only/ mirroring the existing API serving test pattern (@pytest.mark.api, dummy-server based): 2 passed; existing test_api_llmserving.py still green.
• Live test against the real gateway: GET /v1/models → 200 (204 models incl. orcarouter/auto); POST /v1/chat/completions with model="orcarouter/auto" → 200; and the exact APIOrcaRouterServing.generate_from_input path returned "OK" (routed to gemini-3.1-flash-lite).

Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

I'm an engineer on the OrcaRouter team.

[SunnyHaze]

Thanks for the contribution. OrcaRouter already appears to be fully supported through the existing APILLMServing_request, since it exposes an OpenAI-compatible endpoint and does not require provider-specific protocol or response handling.

Adding a dedicated subclass here seems to introduce another abstraction mainly for preset defaults, which may lead to one class per provider over time. I’d prefer keeping the generic serving abstraction and, if useful, adding OrcaRouter as a documented example or introducing a lightweight provider-preset mechanism instead.