greedy decoding

Always pick the single most likely next token (usually a word or word-piece) — same input, same output.

next_token = argmax_i P(token_i | text so far)

After the transformer scores every possible next token, greedy decoding just takes the one with the highest probability, adds it to the text, and repeats. It works because the model's top guess is usually a sensible local choice, and "always pick the best one right now" is the simplest possible rule. But it's shortsighted: the locally-best token can lead into a worse overall sentence, which is why it can sound flat or get stuck repeating. Because there's no randomness, the same prompt always yields the same output.

Appears in

From scores to a word — and the loop Transformers, ELI5 · pt 7

Nearby in the brain

e exponential logits min-p sampling repetition / presence / frequency penalty probability sampling softmax