Browse Teach Curate

Multi-Head Latent Attention and Low-Rank KV Compression

caching a shared low-rank latent instead of per-head keys and values is the current frontier answer; expect this node to move within two quarters

jeremy Video

Multilatent Attention in Transformers Using Low Rank Compression to Reduce Cache Size

Multilatent Attention (MLA) is a mechanism in Transformer architectures that utilizes low-rank compression to project high-dimensional key and value matrices into a compressed latent space, thereby r…