Show HN: 500k+ events/sec transformations for ClickHouse ingestion

Category: infrastructure

Tags: stream-processing, clickhouse, kafka, data-pipeline, etl

Score: 7.0/10 (Innovation: 5, Technical: 8, Documentation: 8, Utility: 7)

GlassFlow is an open-source stream processing engine designed for high-volume data transformation and ingestion from Kafka to ClickHouse, capable of handling over 500k events per second. It's interesting because it combines stateless and stateful transformations, filtering, and dead-letter-queue management into a specialized, performance-focused tool for a specific data pipeline niche.

Target audience: data engineers, devops, backend devs

Repository: https://github.com/glassflow/clickhouse-etl · TypeScript · Apache-2.0 · 446 stars

View on Hacker News