Sparse Autoencoders for Interpreting Neural Network Activations
Using a sparse, overcomplete autoencoder trained to reconstruct the internal activations of a neural network so that its hidden units correspond to a dictionary of human-interpretable, monosemantic features. Unlike a vanilla autoencoder aimed at compression, the sparsity penalty encourages each activation to decompose into a small set of named features, which can then be read off to interpret what a layer represents or clamped to a chosen value to steer the network's behavior. Central to mechanistic-interpretability work on transformer language models.
Scaling Sparse Autoencoders for Interpretability in Large Language Models
Mechanistic interpretability seeks to decompose a neural network's internal activations into human-interpretable units by training a sparse autoencoder (SAE) — a two-matrix model that encodes a dense…