Browse Teach Curate

The Parallel Associative Scan for Training a Linear Recurrence

the algorithm that repays the parallelism debt — an associative recurrence trains in logarithmic depth rather than serially, which is precisely what a nonlinear RNN could never do

jeremy Video

Do we need Attention? A Mamba Primer

Mamba is a class of Linear Time-Varying (LTV) State Space Models designed to address memory and compute inefficiencies inherent in Transformer architectures for long-context language modeling. The co…