Show HN: Cactus v2 – On-device AI with cloud fallback

Category: library

Tags: edge-ai, on-device-inference, hybrid-cloud

Score: 7.8/10 (Innovation: 7, Technical: 9, Documentation: 8, Utility: 7)

Cactus is a hybrid edge-cloud AI engine for mobile devices and wearables, featuring a custom computation graph, zero-copy kernels, and novel rotation-based quantization. It stands out for its seamless cloud fallback mechanism and comprehensive support for multimodal models, enabling high-performance on-device inference with OpenAI-compatible APIs.

Target audience: mobile and embedded developers, ai-ml engineers

Repository: https://github.com/cactus-compute/cactus · C++ · NOASSERTION · 5400 stars

View on Hacker News