Show HN: How-to-Train-Your-GPT

Category: other

Tags: gpt, transformer, deep-learning, tutorial, llm, attention

Score: 7.3/10 (Innovation: 6, Technical: 7, Documentation: 9, Utility: 7)

This project is an interactive textbook that teaches you to build a modern GPT-style language model from scratch, with fully annotated code and explanations. It stands out for its exceptionally clear, analogy-driven documentation and focus on the latest LLaMA 3 architecture techniques like RoPE, RMSNorm, and SwiGLU.

Target audience: backend devs, data engineers

Repository: https://github.com/raiyanyahya/how-to-train-your-gpt · Jupyter Notebook · MIT · 777 stars

View on Hacker News