Show HN: iPhone ANE holds LLM tok/s while MLX and LiteRT thermal-throttle

Category: devtools

Tags: benchmarking, llm, apple-silicon, machine-learning, performance

Score: 6.8/10 (Innovation: 6, Technical: 7, Documentation: 7, Utility: 7)

This project provides a neutral, reproducible benchmarking framework for running LLMs on Apple Silicon devices (iPhone, iPad, Mac). It is interesting because it reveals counterintuitive performance trade-offs between GPU and Neural Engine runtimes, particularly showing that the ANE can outperform GPU-based runtimes in sustained throughput due to superior thermal management.

Target audience: backend devs, data engineers, ml engineers

Repository: https://github.com/john-rocky/apple-silicon-llm-bench · Swift · MIT · 4 stars

View on Hacker News