Router that manages artificial intelligence traffic
Developed by NVIDIA, Switchyard is a high-performance artificial intelligence inference engine written in Rust language. It offers an optimized runtime environment to run large language models (LLM) efficiently on different hardware infrastructures.
Updates
- September 27, 2026: Stars 2,617 → 3,227, latest release v0.3.0 (September 22, 2026).
- August 31, 2026: Stars 1,566 → 2,617, latest release v0.2.0 (August 10, 2026).
- August 15, 2026: Stars 923 → 1,566, latest release v0.2.0 (August 10, 2026).
What you get
- Routing traffic between different artificial intelligence models
- Translation between OpenAI and Anthropic API formats
- Track transaction metrics and error logs
Installation
curl -LsSf https://astral.sh/uv/install.sh | sh
source "$HOME/.local/bin/env"
uv tool install --python 3.10 "nemo-switchyard[cli]"cargo install --locked switchyard-server
switchyard-server --helpRunning it
curl http://localhost:4000/healthIf you don't write code
Act as an AI traffic router for me. Using Switchyard, I want you to distribute the requests of my coding agents like Claude Code or Codex between different models, automatically translate between OpenAI and Anthropic API formats, and monitor all operational metrics. Manage incoming requests with structured routing algorithms and perform A/B testing or load balancing between different models when necessary.
Repository: https://github.com/NVIDIA-NeMo/Switchyard. Follow the official instructions in the README of this repository for setup and execution.
Before starting, check which OS we are on and if the necessary tools (e.g., Python, Node.js, Git, Docker) are installed. Install any missing ones step-by-step according to this system.
Create a new, empty folder to work in. Do not touch any files outside this folder and do not delete anything. Tell me what you are going to do in one sentence before running each command.
Tell me what I should see on the screen at the end of each step. If an error occurs, I will paste the error message to you exactly as it is, and we will fix it together.
Related dictionary terms
Links
TreScout did not build this tool · we found it in GitHub trends and wrote it up. This page describes the repository as of 2026-08-13: The star count and our text belong to that day, the repository may have changed since. Check the repository link for the current state. The introductory text was prepared with AI from the repository's own documentation, and this page was machine-translated from the Turkish original · the Turkish version prevails.