Rust AI Control Plane
publishedPolicy-aware agent orchestration with strong observability and security boundaries
A Rust-centric stack for orchestrating AI agents with Axum for the API gateway, Postgres for state, NATS for inter-agent messaging, and OpenTelemetry for distributed tracing. Designed for teams building production AI systems that need governance, audit trails, and reliable execution.
Component Matrix
Why This Exists
Why This Stack Exists
AI agents in production need more than a Python script and an API key. They need:
- Governance: Policy enforcement on what agents can do
- Observability: Distributed tracing across multi-agent sessions
- Reliability: Rust's memory safety prevents entire classes of runtime failures
- Performance: Low-latency orchestration for real-time agent coordination
This stack was born from building production agent systems where Python's runtime characteristics became a liability.
Tradeoffs
Tradeoffs
- Steep hiring ramp: Finding Rust + AI engineers is genuinely difficult
- Slower iteration: Prototyping in Rust takes longer than Python
- Smaller AI ecosystem: Most ML libraries are Python-first
- Operational complexity: NATS adds a messaging layer to manage
- Not beginner-friendly: The learning curve is real and affects onboarding time
Claims (5)
Axum hits the best ergonomics/control tradeoff for small teams building Rust-native control planes.
SQLx compile-time query checking reduces production incidents from schema drift significantly.
This stack is not beginner-friendly and requires significant Rust experience to be productive.
NATS is a better fit than Kafka for agent messaging at startup scale due to lower operational overhead.
This composition is strong for security-sensitive AI tooling where audit trails matter.
Evidence (6)
docs.rs adopted SQLx because "a common source of outages is an incorrect query with no tests"
moderateThe rust-lang/docs.rs project adopted SQLx specifically because "a common source of outages is an incorrect query with no tests" and SQLx "can check all queries at build time without having to write a test for that specific query." Real production rationale from a critical Rust infrastructure project.
NATS JetStream vs Kafka: 4x less resources, 10-50x lower latency, single binary deploy
strongNATS JetStream: 2+ vCPU / 4GB RAM for production vs Kafka: 8+ vCPU / 16GB RAM (4x resources). Latency: NATS sub-millisecond in-memory, 1-5ms persisted vs Kafka 10-50ms due to batching. Throughput: Kafka leads at 500K-1M+ msg/sec vs NATS 200K-400K — but for agent messaging (not log aggregation), NATS throughput is sufficient. NATS is a single Go binary vs Kafka's multi-broker + ZooKeeper/KRaft complexity.
NATS and Kafka architectural comparison — Synadia
moderateDetailed architectural comparison (vendor blog but thorough). Key insight: "For service-to-service notifications, lightweight task distribution, or real-time command dispatch where messages older than an hour are useless, Kafka's durability model is solving a problem you don't have." NATS built-in clustering, auth, and monitoring vs Kafka's ecosystem of separate tools.
Cloudflare Infire: Rust inference engine — 7% faster than vLLM, 82% less CPU
strongCloudflare built Infire, an LLM inference engine written in Rust. Results: 7% faster than vLLM 0.10.0, uses only 25% CPU vs vLLM's >140% (82% reduction). Cuts CPU overhead via compiled CUDA graphs. Powers Llama 3.1 8B on Cloudflare edge. Real production evidence that Rust is viable for AI infrastructure.
Qdrant: $87.8M funded Rust vector database — Tripadvisor, HubSpot, Canva in production
strongQdrant, written entirely in Rust, raised $50M Series B (total $87.8M, March 2026). 29,762 GitHub stars, 250M+ downloads. Production users: Tripadvisor, HubSpot, OpenTable, Canva, Bosch, Roche. Key claim: "Infrastructure on the critical path of production AI cannot afford garbage collection pauses."
HuggingFace TGI: Rust HTTP server + scheduler for production LLM inference
strongHuggingFace Text Generation Inference uses Rust for the HTTP server and scheduling layers, Python for model execution. 10,811 GitHub stars. Powers HuggingChat and Inference API in production. TGI v3.0 claims 13x faster than vLLM on long prompts. Demonstrates the "Python for models, Rust for orchestration" pattern.