Show HN: Drift – an embedding-model upgrade should be a rotation, not a reindex

Category: infrastructure

Tags: embeddings, rag, spark, vector-database, model-migration, data-lineage

Score: 8.0/10 (Innovation: 8, Technical: 8, Documentation: 9, Utility: 7)

Drift is a Spark-native embedding lifecycle management tool that replaces ad-hoc RAG embedding scripts with declarative commands for embedding, incremental CDC refresh, and model migration. Its key innovation is the Drift-Adapter approach for near-zero-downtime embedding model upgrades using orthogonal Procrustes rotation, supported by a lineage ledger for cost and compliance tracking.

Target audience: data engineers, ml engineers, backend devs

Repository: https://github.com/aayush4vedi/drift-spark · Python · MIT

View on Hacker News