Skip to content

feat(translation): prepare requests for routed targets - #455

Open
afourniernv wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
afourniernv:afournier/switch-1253-target-prompt-translation
Open

feat(translation): prepare requests for routed targets#455
afourniernv wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
afourniernv:afournier/switch-1253-target-prompt-translation

Conversation

@afourniernv

@afourniernv afourniernv commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds one translation-layer helper for preparing a normalized request for a routed target.

This is part 1 of 3 for #496 / SWITCH-1253. It adds the provider-aware operation used by libsy in #463; it does not change router behavior or TOML configuration by itself.

Problem

A decoded request can contain both:

  • Switchyard's normalized request, and
  • an exact provider body retained for lossless same-format forwarding.

Changing only the normalized model is unsafe: exact replay can still send the route alias instead of the selected upstream model.

Behavior

prepare_request_for_target(&mut request.llm_request, target, prompt);
  • Always updates the normalized model.
  • With no target prompt, updates the top-level model in preserved OpenAI Chat, OpenAI Responses, and Anthropic Messages bodies, retaining their provider-only fields.
  • Discards custom or malformed preserved bodies when Switchyard cannot safely update their model.
  • With a target prompt, prepends the normalized system instruction and clears preserved request bodies because they predate the content mutation.

The prompt path intentionally re-encodes from the normalized request; it does not retain or patch a second provider-body copy.

API impact

This PR adds the public Rust function switchyard_translation::prepare_request_for_target(...). It does not change Python APIs, server APIs, router construction, or the TOML schema.

Suggested review

  1. Translation helper and tests
  2. Confirm the no-prompt test both retains a provider-only field and emits the selected model.
  3. Confirm the prompt test invalidates exact replay.

Unique change: one signed commit, 3e63c460 (3 files, +123/-5).

Validation

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • uv run ruff check .
  • uv run mypy switchyard
  • uv run pytest tests/ -v -m "not integration" (113 passed)
  • Strict MkDocs build

Stack

PR Layer Responsibility
#455 Translation Prepare one routed request without stale exact replay
#463 libsy Prepare selected, fallback, and routing-time candidates
#464 Server Add targets.*.system_prompt, compatibility, and integration coverage

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8d3a3bb9-7653-48a1-b20d-074038e4ab19

📥 Commits

Reviewing files that changed from the base of the PR and between c7c07d5 and 7f74985.

📒 Files selected for processing (3)
  • crates/switchyard-translation/src/lib.rs
  • crates/switchyard-translation/src/util.rs
  • crates/switchyard-translation/tests/request_translation.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The translation crate exposes request preparation for target models and optional prompts. It patches preserved OpenAI Chat, OpenAI Responses, and Anthropic bodies, handles custom formats, restores preservation metadata, and adds cross-format and safety tests.

Changes

Request preparation

Layer / File(s) Summary
Preparation API and orchestration
crates/switchyard-translation/src/lib.rs, crates/switchyard-translation/src/util.rs
The crate exports prepare_request_for_target. The function updates the target model, optionally prepends a system instruction, and processes preserved bodies.
Preserved-body patching
crates/switchyard-translation/src/util.rs
Built-in OpenAI Chat, OpenAI Responses, and Anthropic bodies receive model and prompt updates. Unsupported custom bodies are retained without prompts or removed when patching is required. Preservation metadata is restored after serialization.
Preparation behavior validation
crates/switchyard-translation/tests/request_translation.rs
Tests cover target and prompt updates, provider-field preservation, cross-format relaying, no-prompt behavior, and removal of unpatchable bodies.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 7f749

The translation change is localized and preserves provider request fields while applying routed target metadata and prompts. No actionable merge-blocking risk remains beyond normal checks and review.

Poem

I’m a rabbit with a request to prepare,
Models and prompts now travel with care.
OpenAI hops, Anthropic too,
Custom bodies know what they can do.
Tests guard each preserved hare.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding request preparation for routed targets in the translation layer.

Comment @coderabbitai help to get the list of available commands.

@afourniernv
afourniernv force-pushed the afournier/switch-1253-target-prompt-translation branch from 7f74985 to 55d2a22 Compare August 19, 2026 16:29
@afourniernv

Copy link
Copy Markdown
Contributor Author

Tracking issue: #496

Signed-off-by: Alex Fournier <afournier@nvidia.com>
@afourniernv
afourniernv force-pushed the afournier/switch-1253-target-prompt-translation branch from 55d2a22 to 3e63c46 Compare August 24, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant