Uncertainty Estimation in Large Language Models
The set of techniques for quantifying how confident a large language model is in a given output, so that low-confidence answers can be abstained from, flagged, or handled differently. Signals include the cumulative or average token probabilities of a generated answer, the consistency (semantic agreement) across multiple sampled generations, self-evaluation prompts in which the model rates its own certainty, and probes of internal decoding states. Reliable uncertainty estimates underpin calibration, hallucination detection, and preference construction for alignment.
Uncertainty Quantification in Large Language Models using Perturbation and Random Walks
The concept presents a fully probabilistic framework for uncertainty quantification in Large Language Models by defining input and output spaces as random variables governed by joint distributions de…