Show HN: A 155K-param transformer builds a map of a world it's never shown

Category: ai-ml

Tags: transformer, world-model, interpretability

Score: 6.3/10 (Innovation: 8, Technical: 7, Documentation: 3, Utility: 4)

This project demonstrates a tiny 155K-parameter transformer that learns to build an internal map of a procedurally generated world it has never seen, revealing emergent spatial understanding. It is interesting as a minimalist example of how world models can form in small neural networks, with potential insights for interpretability and reinforcement learning.

Target audience: AI researchers, ML engineers

Repository: https://ankur-chr.github.io/inside-the-model/

View on Hacker News