D
Demerzel
Video
Multi-Head Attention
Splitting attention into parallel heads over learned subspaces so different heads capture different relations; concatenation and output projection.