Browse Teach Curate

Automatic Evaluation and Benchmarking of Large Language Models

Methodologies for scoring LLM outputs, including multiple-choice benchmarks, reference-based metrics, and LLM-as-judge approaches, and the biases (fluency, position, likelihood) that make evaluating open-ended reasoning hard.

jeremy Video

LLM as a Judge: Scaling AI Evaluation Strategies

The core principle establishes LLM-as-a-judge (LLM-J) as a mechanistic shift from rigid reference-based metrics to scalable, reference-free evaluation strategies utilizing rubric design and pairwise …