对比分组与Transformer用于指称图像分割
Contrastive Grouping with Transformer for Referring Image Segmentation
AI总结：
本文提出CGFormer，一种基于对比分组与Transformer的掩码分类框架，通过令牌查询和分组策略捕获对象级信息，在指称图像分割中显著优于现有方法。
摘要：
指称图像分割旨在根据自然语言表达对图像中的目标指称进行分割。现有的一阶段方法采用逐像素分类框架，试图在像素级别直接对齐视觉和语言，因此无法捕获关键的对象级信息。在本文中，我们提出了一种掩码分类框架，即对比分组与Transformer网络（CGFormer），通过基于令牌的查询和分组策略显式捕获对象级信息。具体来说，CGFormer首先引入可学习的查询令牌来表示对象，然后交替查询语言特征并将视觉特征分组到查询令牌中，以进行对象感知的跨模态推理。此外，CGFormer通过每两个连续层中联合更新查询令牌和解码掩码来实现跨层交互。最后，CGFormer将对比学习与分组策略相结合，以识别与指称对应的令牌及其掩码。实验结果表明，CGFormer在分割和泛化设置中均持续且显著地优于最先进的方法。
英文摘要：
Referring image segmentation aims to segment the target referent in an image conditioning on a natural language expression. Existing one-stage methods employ per-pixel classification frameworks, which attempt straightforwardly to align vision and language at the pixel level, thus failing to capture critical object-level information. In this paper, we propose a mask classification framework, Contrastive Grouping with Transformer network (CGFormer), which explicitly captures object-level information via token-based querying and grouping strategy. Specifically, CGFormer first introduces learnable query tokens to represent objects and then alternately queries linguistic features and groups visual features into the query tokens for object-aware cross-modal reasoning. In addition, CGFormer achieves cross-level interaction by jointly updating the query tokens and decoding masks in every two consecutive layers. Finally, CGFormer cooperates contrastive learning to the grouping strategy to identify the token and its mask corresponding to the referent. Experimental results demonstrate that CGFormer outperforms state-of-the-art methods in both segmentation and generalization settings consistently and significantly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
