Show HN: Do you want a semantic cache for free with zero code changes?
Category: infrastructure
Tags: semantic-cache, llm, redis, python, caching
Score: 7.3/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 7)
Khazad is a transparent transport-layer semantic cache for LLM API calls, using Redis Vector Sets to intercept HTTP calls and serve semantically equivalent cached responses without code changes. It's interesting because it reduces API costs and latency dramatically with zero application modifications, supporting multiple major LLM providers and handling streaming responses.
Target audience: backend devs
Repository: https://github.com/GuglielmoCerri/khazad · Python · MIT · 7 stars
View on Hacker News