Learn · Autoregressive Generation
Autoregressive Generation
A decoder language model doesn't write a sentence all at once — it writes one token at a time. Each token it produces is fed back in as part of the input for the next step. Step through the loop below and watch the text grow.
Seed prompt:
The robot learned to ___
The next-token candidates below are illustrative — a small, fixed toy model keyed on the last token. The maths (softmax over logits, weighted random sampling) is real.
Scales the logits before softmax. Low = focused and repetitive, high = more surprising.
1 · Context fed back into the model
Everything generated so far becomes the input for the next prediction. The highlighted last token is the key this toy model conditions on.
2 · Next-token probability distribution
softmax(logit / temperature) turns the toy model's raw scores into a
probability for each candidate token.
3 · The growing sequence
One token is sampled from the distribution above and appended. Each token is tagged with the step that produced it; the seed prompt is step 0.
What's happening
What "autoregressive" means
Every token is predicted conditioned on all the previous tokens — including
the ones the model just generated itself. The output at step t becomes
part of the input at step t+1. That feedback loop is the whole game.
Why causal attention
To condition on "all previous tokens" without peeking ahead, decoders use causal self-attention: each position can attend only to earlier positions. That masking is exactly what makes left-to-right generation well-defined.
Temperature & sampling
The same logits can yield very different text. Temperature and sampling strategy decide how boldly the model picks — explore that in the Temperature & Sampling demo.
This is a toy: a real model conditions each prediction on the entire context through stacked attention layers and has a vocabulary of tens of thousands of tokens, not a handful keyed on the last word. For related demos, see Temperature & Sampling, Causal Attention, or the Learn page.