Tiny LLM Lab
A language model small enough to see every weight. Real ones have billions; this one has about a hundred, so you can watch it learn and see exactly how it picks the next word.
- MODEL
- ≈ 100 weights, trained in your browser
- LANGUAGE
- HTML · JS · Python
- FRAMEWORK
- None (numpy for the Python version)
- MEMORY
- Weights file · JSON or safetensors
- LAST PUSH
- 1 Nov 2026
- STARS
- ★ 0
Train, save, predict.
- 01 · TOKENSSplit text into tokens and IDs
- 02 · TRAINNudge random weights toward the real next token
- 03 · SAVEWrite the weights to a file
- 04 · SOFTMAXFreeze them and turn scores into chances
- 05 · PICKChoose a token
Every language model does the same three things. It trains: start from random weights and nudge them so the real next token gets a higher score. It saves the learned weights to a file. Then it freezes them and predicts: turn the scores into chances with softmax, pick a token, and repeat.
Here the training rule is about ten lines of plain code: guess, measure how far off the guess was, and move the weight by a fraction of that. Real models add subword tokens, attention over the whole input and billions of weights, but the output step and the training rule are the same.
This is the hands-on lab for Episode 2 of Zero to AI Agent, “How an LLM actually thinks”. Everything runs in the browser: no server, no install, no AI library.
- 01Tiny LLM LabTrain on any text, watch the weight grid change, save the weights, then predict the next token with temperature and top-k.
- 02Text to numbersYour sentence in six steps, from characters and byte-pair tokens to IDs, embeddings and the input grid.
- 03TransformerA real one-block transformer trained in the browser. Follow a prompt through every block and see where it beats the simple model.
- 04How a weight is madeOne number, from random to useful. Like salting a curry, taste, see how far off, add a pinch.
git clone https://github.com/kagitha-dev/tiny-llm-lab
cd tiny-llm-lab
pip install numpy
python tiny_llm.py train data.txt --rounds 50
python tiny_llm.py predict "the dog" # dog -> sat 96%Build AI agents in Java from scratch. LLMs, Ollama, tool calling, MCP, RAG and LangChain4j, with slides and hands-on labs for every episode.
See the series →