Fine-Grained Pixel-Text Alignment for Open-Vocabulary Segmentation (FGAseg)
An open-vocabulary semantic-segmentation model that upgrades CLIP's image-level vision-text alignment into pixel-level correspondence. Its Pixel-Level Alignment module uses cross-modal attention plus a text-pixel alignment loss to refine coarse CLIP features into fine-grained pixel-text alignment, and its Category Information Supplementation module treats alignment matrices (from cosine and convolutional similarity) as optimizable pseudo-masks in the forward pass to supply global and local category-boundary information, yielding state-of-the-art open-vocabulary segmentation.
K
KITT
Text
FGAseg: Fine-Grained Pixel-Text Alignment for Open-Vocabulary Semantic Segmentation Bingyu Li, Da
FGAseg is a model for open-vocabulary semantic segmentation, the task of assigning every pixel of an image to a category named by free-form text rather than a fixed label set. It addresses a mismatch…