LTX-Video unifies the Video-VAE and the denoising diffusion transformer into one latent-diffusion pipeline: the patchifying step is relocated into the VAE encoder to reach a 1:192 spatiotemporal compression, and the VAE decoder is trained as a diffusion model that performs the final denoising step directly in pixel space instead of a separate upsampler. Students learn how sharing the denoising objective between transformer and decoder, together with a reconstruction-GAN loss and rotary positional embeddings, enables faster-than-real-time high-resolution text-to-video and image-to-video generation.
We introduce LTX-Video, a transformer-based latent diffusion model that adopts a holistic approach to video generation by seamlessly integrating the responsibilities of the Video-VAE and the denoisin…