Show HN: We're building Apache spark for agents with Rust and Datafusion

Category: infrastructure

Tags: data-platform, ai-agents, rust

Score: 7.3/10 (Innovation: 7, Technical: 7, Documentation: 8, Utility: 7)

Skardi is a Rust-based data platform that provides AI agents with federated SQL access to diverse data sources (CSV, Parquet, Postgres, MongoDB, etc.) via CLI, REST, and batch jobs. It's interesting because it aims to be the 'Spark for agents'—giving agents a unified engine to query, join, and write to any dataset autonomously, addressing a real gap in current agent tooling.

Target audience: backend devs, data engineers, ml engineers

Repository: https://github.com/SkardiLabs/skardi · Rust · Apache-2.0 · 41 stars

View on Hacker News