Skip-Connection Feature Modulation for Concept Erasure in Diffusion Models
DuMo erases a target concept from a text-to-image diffusion model without retraining or editing the backbone. The U-Net is frozen and an Eraser with Prior Knowledge module edits its skip-connection features, so erasure lands mainly on high-frequency image detail while low-frequency structure and non-target concepts are preserved, using the original skip features as a prior. A Time-Layer Modulation process then scales erasure strength per layer and diffusion timestep to balance removal against generation quality.
DuMo: Dual Encoder Modulation Network for Precise Concept Erasure Feng Han1,2, Kai Chen1,2, Chao
DuMo (Dual encoder Modulation network) is a method for erasing a target concept (e.g. NSFW content or copyrighted style) from a text-to-image diffusion model while minimally disturbing unrelated conc…