Conceptual

Protein Language Models: ESM and Learned Sequence Representations

Training a transformer to fill in masked residues across evolutionary-scale protein data yields embeddings that encode structure and function without ever being shown a structure. These representations transfer to variant effect, function, and single-sequence folding tasks.