💬 tinytalk

A 14M-parameter GPT written from scratch in PyTorch — hand-written attention, custom BPE tokenizer, no pretrained weights.

13.9M params 6 layers · 6 heads 256 token context perplexity 17.8 trained in 41 min

Trained for 41 minutes on everyday dialogue, so it manages small talk and little else — it has no facts and forgets past ~4 turns.
Source and write-up on GitHub →