Dynamics-Aware Token Pruning for Diffusion Model Acceleration
A training-free method (DaTo) that unifies feature caching and token pruning to accelerate diffusion-model inference. Observing that feature caching makes activations across adjacent denoising timesteps too similar and flattens their temporal dynamics, DaTo selectively prunes low-dynamics tokens so that only high-dynamics tokens pass through the self-attention layers, restoring feature dynamics across timesteps. It achieves both temporal and token-wise reuse, delivering roughly 9x speedup on Stable Diffusion while improving FID.
Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free
Stable Diffusion has achieved remarkable success in the field of text-to-image generation, with its powerful generative capabilities and diverse generation results making a lasting impact. However, i…