That is all a language model ever does: guess one word, add it, guess again. Type the start of a sentence, then open this small one up and see every step behind the guess.
Or try
Preparing the model…
The model has read thousands of short sentences and knows a few thousand words. This is the sort of thing it picked up. In grey: what it answers.
A teaching model, not a product. Here is exactly how far it goes.
Embeddings, attention, masking, softmax, layer normalisation and sampling all work the way they do in GPT-style models. They are just small enough here that every number fits on screen, and you can click any of them to see how it was worked out.
A real model learns by adjusting billions of weights. This one has random weights, so its knowledge comes from counting which word followed which phrase in its reading. Turn “Training-text strength” down to zero in the settings and you will see the network on its own: it babbles.
It repeats what it has read and falls apart outside it. Ask it for the capital of France and it knows; ask it why, and it does not. That is the point: it is small enough to watch, one step at a time.
Change how the model thinks and how it chooses. Tap i to learn what each setting does.