Dual Diffusion Transformer for Unified Vision-Language Generation and Understanding
This work builds a single dual-branch diffusion transformer that both generates and understands images, using continuous diffusion for image latents and discrete diffusion for text tokens, trained under one cross-modal maximum-likelihood objective. Students learn how a joint diffusion objective back-propagated through both modalities yields one model capable of text-to-image generation, captioning, and visual question answering, positioning multimodal diffusion as an alternative to autoregressive next-token prediction.
2501.00289
Presents a single end-to-end diffusion model that both generates and understands images, closing the gap with autoregressive vision-language models. Built on the multimodal diffusion transformer (MM-…