Browse Teach Curate

Temperature and Sampling in Language Model Decoding

How a language model turns its next-token probability distribution into text: greedy, temperature-scaled, top-k and nucleus sampling. The temperature parameter rescales the logits, trading determinism for diversity, so it directly controls the reproducibility and variability of a model's outputs.

jeremy Video

What is Temperature in LLM Sampling

In large language model inference, text generation is probabilistic, governed by sampling distributions derived from softmax-processed logits where temperature acts as a scalar modulator controlling …