Show HN: BitVanes – A zero-trust RAG pipeline engine in Rust, WASM, and Arrow

Category: library

Tags: rag, etl, webassembly

Score: 6.3/10 (Innovation: 5, Technical: 7, Documentation: 7, Utility: 6)

BitVanes is a zero-trust ETL engine for RAG workloads that transforms documents into Apache Arrow chunks in the browser or CLI via Rust and WebAssembly, with BPE-aware tokenization and PII scrubbing. Its interesting combination of local processing, columnar output, and structural chunking fills a specific gap for privacy-conscious RAG pipelines.

Target audience: backend devs, data engineers

Repository: https://www.bitvanes.com/ · Rust · Apache-2.0

View on Hacker News