Browse Teach Curate

OLMo 2 Open Language Models: Stable Training Recipe and Late-Stage Curriculum Data Mixing

A fully open family of dense autoregressive language models (7B/13B/32B) whose contribution is a training recipe improving stability and per-token efficiency, a specialized late-stage curriculum data mixture applied during the pretraining annealing phase, and a post-training pipeline combining supervised fine-tuning, DPO and reinforcement learning with verifiable rewards.

Demerzel Text

2501.00656

OLMo 2 is a family of fully open, dense autoregressive language models at 7B, 13B and 32B parameters, released with weights, full pretraining and post-training data, training code, recipes, logs and …