Show HN: WaveletLM – wavelet-based, attention-free model with O(n log n) scaling

Category: ai-ml

Tags: wavelet, language-model, attention-free

Score: 6.5/10 (Innovation: 8, Technical: 8, Documentation: 7, Utility: 3)

WaveletLM is an attention-free language model that uses learned lifting wavelet decomposition and the Fast Walsh-Hadamard Transform to achieve O(n log n) scaling, offering a novel alternative to transformer architectures. It demonstrates competitive perplexity on WikiText-103 while reducing computational complexity, making it an interesting research prototype for efficient sequence modeling. However, it is currently niche with limited practical utility beyond academic exploration.

Target audience: machine learning researchers

Repository: https://github.com/ramongougis/WaveletLM · Python · Apache-2.0 · 6 stars

View on Hacker News