Show HN: Suture – a reverse proxy that repairs truncated JSON in LLM streams

Category: infrastructure

Tags: reverse-proxy, llm, json-repair, streaming, rust

Score: 7.5/10 (Innovation: 7, Technical: 8, Documentation: 8, Utility: 7)

Suture is an ultra-low-latency reverse proxy written in Rust that repairs truncated JSON in LLM streaming responses on the fly, solving a common pain point where tool calls or structured outputs break due to max_tokens or dropped connections. Its innovative combination of a state-machine-based repair engine, SSE awareness, and per-provider delta reconstruction makes it a practical drop-in fix for a frustrating problem.

Target audience: backend devs, data engineers, devops

Repository: https://github.com/tensorhq/suture-stream-repair · Rust · Apache-2.0 · 1 stars

View on Hacker News