Show HN: Sturnus – OpenAI-compatible LLM proxy routing to the fastest provider

Category: infrastructure

Tags: llm-proxy, latency-routing, openai-compatible

Score: 6.8/10 (Innovation: 5, Technical: 7, Documentation: 8, Utility: 7)

Sturnus is a lightweight Rust sidecar that routes LLM API requests to the fastest available provider based on real-time latency and error rates, exposing an OpenAI-compatible endpoint. It offers a novel latency-aware routing algorithm with session affinity and memory-bounded operation, all in a single static binary with no external dependencies. This is interesting for teams needing production-grade multi-provider LLM routing without the complexity of larger gateways.

Target audience: backend devs

Repository: https://github.com/sturnus-dev/sturnus · Rust · MIT · 3 stars

View on Hacker News