Show HN: Cachet – A drop-in semantic cache for LLM APIs, 100% local, in Rust

Category: infrastructure

Tags: llm-caching, rust, semantic-cache, api-proxy, cost-saving

Score: 7.0/10 (Innovation: 6, Technical: 7, Documentation: 8, Utility: 7)

Cachet is a lightweight, Rust-based semantic caching proxy for LLM APIs that reduces costs by serving cached responses for semantically similar prompts locally. It combines exact and semantic matching with a live savings dashboard, offering a drop-in replacement for OpenAI/Anthropic base URLs with single-binary deployment.

Target audience: backend devs, ai-engineers, devops

Repository: https://github.com/abhix2112/Cachet · Rust · Apache-2.0

View on Hacker News