HydraRec: Linear-Attention Transformer for Sequential Recommendation
An efficient encoder-based Transformer for sequential recommendation that replaces quadratic dot-product self-attention with 'Hydra attention' — a linear-complexity attention (linear in both sequence length and embedding dimension, adapted from vision transformers) — to handle long user-interaction histories and growing catalogues while preserving temporal context. With causal masking HydraRec outperforms both dot-product and other linear-attention recommendation models on next-item prediction; in bidirectional form it matches BERT4Rec with faster running time.
Accepted at Computing Conference 2025
Introduces HydraRec, an efficient Transformer-based model for sequential recommendation (next-item / next-basket prediction from a user's interaction history). Standard dot-product self-attention in …