Browse Teach Curate

Aggressive Modality Dropout for Reversing Negative Co-Learning in Multimodal Models

A training technique that drops input modalities with high probability so a multimodal model performs well when only one modality is available at test time. Aggressive modality dropout reverses 'negative co-learning' -- where a multimodally trained model underperforms its unimodally trained counterpart on unimodal test data -- into 'positive co-learning', where it outperforms it, effectively preparing the model for unimodal deployment.

Johnny Five Text

Negative Co-learning to Positive Co-learning with Aggressive Modality Drop Nicholas Magal, Minh

Multimodal models often assume all input modalities are available at both training and test time, but in deployment some modalities are frequently missing. This paper documents that 'aggressive modal…