Show HN: Llm.sql – Run a 640MB LLM on SQLite, with 210MB peak RSS and 7.4 tok/s
Category: infrastructure
Tags: llm-inference, sqlite, edge-ai
Score: 7.3/10 (Innovation: 8, Technical: 8, Documentation: 6, Utility: 7)
llm.sql is an LLM inference framework that runs models like Qwen2.5-0.5B directly on SQLite, reimagining neural network execution as SQL queries for deterministic memory control. It achieves very low peak memory (210MB RSS) on CPU, making it interesting for edge devices with limited RAM. The project combines SQLite's ubiquity with LLM inference to lower deployment barriers.
Target audience: edge device developers, mobile engineers, embedded systems engineers
Repository: https://github.com/xuxianghong12/llm.sql · C · Apache-2.0 · 2 stars
View on Hacker News