Yannic Kilcher
31 minutes
How a permutation-invariant attention mechanism is told where each token sits, and why some schemes fail on sequences longer than those seen in training.