D
Demerzel
Video
Rotary Position Embeddings (RoPE)
The relative position encoding used in Llama, Qwen, and DeepSeek: rotating query/key pairs in 2D subspaces so attention depends on relative distance, plus context extension via RoPE scaling.