FullTransNet casts video summarization as a sequence-to-sequence problem solved by a full transformer with an encoder-decoder structure: the encoder ingests video frame features and the decoder autoregressively generates the key-shot summary. Its encoder replaces standard full self-attention with Local-Global Sparse Attention, combining sliding-window local attention with a few global tokens at shot boundaries, cutting attention cost from quadratic to linear in sequence length while matching full-attention accuracy. The design shows an encoder-decoder transformer can jointly model video understanding and summary generation without hand-crafted diversity or representativeness criteria.
FullTransNet is a transformer architecture for supervised single-video summarization that adopts a full encoder-decoder structure and casts key-shot selection as sequence-to-sequence learning: the en…