NVIDIA introduces switchyard: a rust tool for routing and translating LLM traffic between major apis

NVIDIA has launched Switchyard, a Rust-based proxy and library designed to route and translate large language model (LLM) traffic between OpenAI and Anthropic APIs. The tool allows teams to seamlessly switch between different LLM providers without rewriting their coding agents, as each provider uses a distinct API format. Switchyard translates between these formats, records operational metrics, and offers composable routing algorithms. It is released under the Apache 2.0 license and is available at docs.nvidia.com/nemo/switchyard. While Switchyard is deployable for evaluation purposes, NVIDIA has labeled it as pre-alpha and experimental, warning that it is not suitable for production use. The tool supports three inbound formats—OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages—and allows clients to maintain their native API while routing requests to the appropriate backend. The tool is available through crates.io and PyPI, with separate installation paths for the server, launcher, and library components. Switchyard also provides detailed metrics through Prometheus and OpenTelemetry, including routing overhead and session statistics. The tool is intended for developers and teams looking to integrate multiple LLM providers efficiently, with deployment options through TOML files and environment variables for secure credential handling.