Object-Level Visual Prompts for Compositional Image Generation
Guiding a pretrained text-to-image diffusion model with object-level visual prompts (foreground objects plus background) instead of, or alongside, text, to compose semantically coherent scenes without a predefined layout. The central mechanism is a KV-mixed cross-attention that draws keys (layout, from a small-bottleneck encoder) and values (fine appearance, from a larger-bottleneck encoder) from complementary visual representations, preserving each object's identity while allowing diverse arrangement and pose; object-level compositional guidance at inference further improves identity and layout fidelity.
Object-level Visual Prompts for Compositional Image Generation Gaurav Parmar1,2 Or Patashnik2,3
This paper introduces a method for composing object-level visual prompts (foreground objects plus background) inside a pretrained text-to-image diffusion model, so that visual prompts guide generatio…