Show HN: Catch your local LLM falling back to CPU

Category: devtools

Tags: llm-benchmark, gpu-diagnostic, local-llm

Score: 6.8/10 (Innovation: 7, Technical: 6, Documentation: 7, Utility: 7)

Picchio is a diagnostic tool for local LLM setups that benchmarks token generation speed while detecting silent CPU fallback, GPU layer offloading issues, and distinguishing between prefill, decode, and wallclock rates. Its novel three-lane reporting and ability to interrogate posted tok/s numbers address a real blind spot in LLM benchmarking, making it a practical utility for anyone running local models.

Target audience: backend devs, data engineers, devops

Repository: https://github.com/logxio/picchio · Python · MIT · 1 stars

View on Hacker News