Multiscale Multi-Head Attention in Video Transformers for Gesture Recognition
A video transformer for dynamic hand-gesture recognition in which the multi-head self-attention is made multiscale: each attention head operates at a different attention dimension, and the heads are arranged as a pyramidal hierarchy so the network attends to spatio-temporal features at several scales at once. Per-frame spatial features are supplied by a pre-trained backbone network, and the model additionally fuses several sensor modalities - RGB, depth, infrared, surface normals derived from depth maps, and optical flow - for single-modal and multimodal recognition.
Multiscaled Multi-Head Attention-based Video Transformer Network for Hand Gesture Recognition
This paper introduces MsMHA-VTN, a video transformer network for recognizing dynamic hand gestures. Its key idea is a multiscaled multi-head attention: rather than giving every attention head the sam…