refactor(relay): migrate dynamic plugin to switchyard-runner

CHANGELOG.md

@@ -8,6 +8,16 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Added
 
+- **NeMo Relay native plugin** — a dynamically loaded integration that loads
+  Switchyard's standard TOML deployment and executes its named routes in
+  process. Managed calls require NeMo Relay 0.8.0 or newer; unknown models use
+  Relay's continuation unchanged.
+
+- **NeMo Relay routing marks** — routing-model usage, measured routing
+  overhead, and selected-model decisions are emitted as ATOF marks. The final
+  serving call remains represented only by Relay's outer LLM lifecycle event to
+  prevent double-counting.
+
 - **Advisor-gate routing** — new `advisor` route type pairing the serving
   executor with a stronger judge-only advisor that reviews terminal turns:
   APPROVE releases the buffered turn, REDO discards it and feeds the advisor's

Cargo.toml

@@ -9,6 +9,7 @@ members = [
     "crates/prefill-router",
     "crates/switchyard-py",
     "crates/protocol",
+    "crates/switchyard-nemo-relay-plugin",
     "crates/switchyard-runner",
     "crates/switchyard-server",
     "crates/switchyard-skill-distillation",
@@ -33,6 +34,7 @@ http = "1"
 httpdate = "1"
 jsonschema = { version = "0.49.4", default-features = false }
 jsonptr = { version = "0.8.1", default-features = false, features = ["std", "json", "resolve"] }
+nemo-relay-plugin = { git = "https://github.com/NVIDIA/NeMo-Relay.git", tag = "0.8.0-rc.3" }
 parking_lot = "0.12"
 rand = "0.10"
 regex = "1"

README.md

@@ -21,6 +21,7 @@ algorithm you write yourself.
 - **Protocol Translation**: convert between OpenAI Chat, Anthropic Messages, and OpenAI Responses formats
 - **Multi-Backend Routing**: random routing, LLM-as-classifier routing, signal-driven stage-router, or your own algorithm
 - **Operational Metrics**: Prometheus metrics cover requests, errors, latency, tokens, and routing overhead
+- **NeMo Relay Plugin**: run random, classifier, escalation, or stage routing in Relay while Switchyard owns provider HTTP dispatch
 
 ## Maturity
 
@@ -123,6 +124,7 @@ configured LLM client selects one upstream format.
 - **[`switchyard-libsy`](crates/libsy/README.md)**: embed routing algorithms in a Rust application
 - **[`switchyard-protocol`](crates/protocol/README.md)**: provider-neutral request, response, and streaming types
 - **[`switchyard-translation`](crates/switchyard-translation/README.md)**: request, response, and stream translation
+- **[`switchyard-nemo-relay-plugin`](crates/switchyard-nemo-relay-plugin/README.md)**: install Switchyard as a native NeMo Relay plugin
 
 ## Community
 

crates/switchyard-nemo-relay-plugin/Cargo.toml

@@ -0,0 +1,28 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+[package]
+name = "switchyard-nemo-relay-plugin"
+version.workspace = true
+description = "Switchyard-owned HTTP routing plugin for NeMo Relay"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+publish = false
+
+[lib]
+crate-type = ["cdylib"]
+
+[dependencies]
+futures-util.workspace = true
+http.workspace = true
+nemo-relay-plugin.workspace = true
+serde.workspace = true
+serde_json.workspace = true
+switchyard-llm-client.workspace = true
+switchyard-protocol.workspace = true
+switchyard-runner.workspace = true
+switchyard-translation.workspace = true
+tokio.workspace = true

crates/switchyard-nemo-relay-plugin/README.md

@@ -0,0 +1,56 @@
+# Switchyard NeMo Relay Plugin
+
+`switchyard-nemo-relay-plugin` is a native NeMo Relay dynamic plugin. It loads
+a standard Switchyard TOML deployment and executes its configured routes in
+Relay through `switchyard-runner`.
+
+The plugin does not define a second routing or target configuration language.
+`switchyard-server` and Relay therefore use the same targets, client pooling,
+algorithm construction, retry policy, and route validation.
+
+## Install
+
+Build the platform bundle with the package script, then configure Relay to load
+the generated `relay-plugin.toml` manifest. The plugin requires NeMo Relay
+`>=0.8.0,<1.0`.
+
+## Configure Relay
+
+Point the dynamic plugin configuration at an existing Switchyard deployment:
+
+```toml
+[[plugins.dynamic]]
+plugin_id = "nvidia.switchyard"
+
+[plugins.dynamic.config]
+priority = 0
+deployment_path = "/etc/switchyard/routes.toml"
+```
+
+`deployment_path` is a Switchyard version-1 TOML deployment, accepted by both
+`switchyard-server` and `switchyard-runner`. See the
+[server configuration guide](../switchyard-server/CONFIGURATION.md) for the
+deployment schema and routing algorithms.
+
+## Request handling
+
+For OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages calls,
+the plugin decodes the Relay request and checks the requested model against the
+deployment's route IDs.
+
+- A configured route is executed by `switchyard-runner`.
+- An unknown model calls Relay's continuation unchanged.
+- The returned provider response is encoded back into the caller's wire format.
+- Streaming responses are returned as unpolled translated streams; Relay owns
+  cancellation and the outer serving-call lifecycle.
+
+The plugin emits a routing request mark, routing-model usage marks, measured
+routing-overhead marks, and a selected-model decision mark. Answer-call usage
+continues to belong to Relay's outer LLM lifecycle.
+
+## Failure policy
+
+`switchyard-llm-client` owns provider retry and route-candidate fallback
+behavior. The plugin does not maintain a separate trusted-default target or
+rerun routing after an execution failure. Failures outside the shared runner,
+including response translation failures, are returned to Relay.

crates/switchyard-nemo-relay-plugin/config.schema.json

@@ -0,0 +1,20 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "Switchyard NeMo Relay Plugin",
+  "description": "Runs a Switchyard deployment through its shared in-process runner.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["deployment_path"],
+  "properties": {
+    "priority": {
+      "type": "integer",
+      "default": 0,
+      "description": "NeMo Relay execution-intercept priority."
+    },
+    "deployment_path": {
+      "type": "string",
+      "minLength": 1,
+      "description": "Path to a Switchyard version-1 TOML deployment shared with switchyard-server."
+    }
+  }
+}

crates/switchyard-nemo-relay-plugin/relay-plugin.toml

@@ -0,0 +1,31 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+manifest_version = 1
+
+[plugin]
+id = "nvidia.switchyard"
+kind = "rust_dynamic"
+
+[compat]
+relay = ">=0.8.0,<1.0"
+native_api = "1"
+
+[defaults]
+enabled = false
+
+[capabilities]
+items = ["plugin_native", "config_schema"]
+
+[config_schema]
+path = "config.schema.json"
+
+[source]
+artifact = "<platform-library-file>"
+
+[integrity]
+sha256 = "sha256:<artifact-sha256>"
+
+[load]
+library = "<platform-library-file>"
+symbol = "nemo_relay_register_plugin"