Browse Teach Curate

Teacher Forcing

During training the decoder is fed the ground-truth previous token instead of its own prediction, making each step a supervised classification and training parallel over positions. The cost is exposure bias: at inference the model consumes its own, possibly wrong, outputs.

jeremy Video

Teacher Forcing in Neural Machine Translation

Teacher forcing is a supervised learning mechanism in sequence-to-sequence models that accelerates training by replacing autoregressive predicted tokens with ground truth labels during the decoder's …