Browse Teach Curate

Full Multi-Head Attention Where the KV Cache Is Not the Constraint

multi-head attention is still the training-time formulation and still correct for short-context and small models — grouping is an inference concession

jeremy Video

Multi-Head Self-Attention Mechanism in Transformer Neural Networks

Visualizes query/key/value projections, scaled dot-product attention, and why running several attention heads in parallel subspaces lets each head learn a different relationship before concatenation …