Superseded by the transformer: the recurrent encoder and decoder must run one timestep at a time, so training cannot be parallelised across the sequence, and attention removed that constraint entirely
Superseded by the transformer: the recurrent encoder and decoder must run one timestep at a time, so training cannot be parallelised across the sequence, and attention removed that constraint entirely
Recurrent Sequence-to-Sequence Translation with a Single Context Vector
the concrete system that defined neural machine translation for several years, and the one the transformer paper was written against
J
jeremy
Video
Seq2seq Encoder-Decoder Models and Context Vector Bottleneck in Neural Machine Translation
The sequence-to-sequence (Seq2seq) architecture operates on a principle where information flow is mediated through a fixed-size context vector derived from the final hidden state of an encoder Recurr…