用于指代表图像分割的协同位置推理网络
Collaborative Position Reasoning Network for Referring Image Segmentation
AI总结：
针对指代表图像分割中实体定位困难的问题，本文提出CPRN，通过RoCo与Holi模块协同建模位置推理，并在三个数据集上持续优于现有方法。
摘要：
给定一幅图像和一个自然语言表达作为输入，指代表图像分割的目标是分割出该表达所指实体的前景掩码。现有方法主要关注视觉与语言之间的交互式学习，以增强用于全局上下文推理的多模态表示。然而，直接在像素级空间中进行预测会导致定位崩溃和较差的分割结果。其主要挑战在于如何显式地建模实体定位，尤其是对于非显著实体。本文中，我们通过所提出的新颖的行-列交互（Row-and-Column interactive，RoCo）模块和引导式整体交互（Guided Holistic interactive，Holi）模块来执行协同位置推理网络（Collaborative Position Reasoning Network，CPRN），从而解决该问题。具体而言，RoCo将视觉特征聚合为分别对应两个方向轴的按行特征和按列特征。它提供了一种细粒度匹配行为，可感知语言特征与两个解耦视觉特征之间的关联，以在层级空间上执行位置推理。Holi通过跨模态注意力机制整合两种模态的特征，并在来自RoCo的定位信息引导下抑制不相关的冗余。因此，通过结合RoCo和Holi模块，CPRN捕获位置推理的视觉细节，使模型能够实现更准确的分割。据我们所知，这是首个显式聚焦于位置推理建模的工作。我们还在三个评估数据集上验证了所提出的方法。该方法始终优于现有的最先进方法。
英文摘要：
Given an image and a natural language expression as input, the goal of referring image segmentation is to segment the foreground masks of the entities referred by the expression. Existing methods mainly focus on interactive learning between vision and language to enhance the multi-modal representations for global context reasoning. However, predicting directly in pixel-level space can lead to collapsed positioning and poor segmentation results. Its main challenge lies in how to explicitly model entity localization, especially for non-salient entities. In this paper, we tackle this problem by executing a Collaborative Position Reasoning Network (CPRN) via the proposed novel Row-and-Column interactive (RoCo) and Guided Holistic interactive (Holi) modules. Specifically, RoCo aggregates the visual features into the row- and column-wise features corresponding two directional axes respectively. It offers a fine-grained matching behavior that perceives the associations between the linguistic features and two decoupled visual features to perform position reasoning over a hierarchical space. Holi integrates features of the two modalities by a cross-modal attention mechanism, which suppresses the irrelevant redundancy under the guide of positioning information from RoCo. Thus, with the incorporation of RoCo and Holi modules, CPRN captures the visual details of position reasoning so that the model can achieve more accurate segmentation. To our knowledge, this is the first work that explicitly focuses on position reasoning modeling. We also validate the proposed method on three evaluation datasets. It consistently outperforms existing state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
