Browse Teach Curate

Multimodal Enrolment Vectors for Target-Speaker Separation

A transformer-based target-speaker separation system (VoiceVector) whose enrolment network produces speaker embeddings from audio-only, audio-visual (lip movements), or silent-video visual data, and whose separation network can be conditioned on multiple positive and negative enrolment vectors, removing the reliance on clean reference audio.

KITT Text

VOICEVECTOR: MULTIMODAL ENROLMENT VECTORS FOR SPEAKER SEPARATION Akam Rahimi, Triantafyllos

VoiceVector is a transformer-based system for separating a target speaker's voice from other speakers and ambient noise. It uses two neural networks: an enrolment network that crafts speaker-specific…