面向SAM辅助少样本分割的前景覆盖原型生成与匹配
Foreground-Covering Prototype Generation and Matching for SAM-Aided Few-Shot Segmentation
AI总结：
本文提出前景覆盖原型生成与匹配方法，利用SAM编码器特征和ResNet特征构建类别一致的查询原型并与支持原型匹配，在多个少样本分割数据集上取得最先进性能。
摘要：
我们提出前景覆盖原型生成与匹配（Foreground-Covering Prototype Generation and Matching）来解决少样本分割（Few-Shot Segmentation，FSS）问题，该任务旨在基于带标签的支持图像分割未标注查询图像中的目标区域。与以往通常利用支持原型和查询像素来估计查询中目标区域的研究不同，我们利用支持原型与查询原型之间的关系。为此，我们利用两种互补特征：用于像素聚合的SAM图像编码器（SAM Image Encoder）特征和用于类别一致性的ResNet特征。具体而言，我们使用SAM特征构建支持和查询原型，并基于ResNet特征区分查询中目标区域的原型。在查询原型构建过程中，我们首先使用传统伪掩码粗略引导SAM特征中的前景区域，然后采用迭代交叉注意力将前景特征聚合到可学习token中。在此过程中，我们发现交叉注意力权重能够有效替代传统伪掩码。因此，我们使用基于注意力的伪掩码引导ResNet特征聚焦于前景，然后将引导后的ResNet特征注入可学习token中，以生成类别一致的查询原型。支持原型的生成与查询原型对称进行，其中伪掩码替换为真实掩码。最后，我们将这些查询原型与支持原型进行比较以生成提示，随后通过SAM掩码解码器（SAM Mask Decoder）生成目标掩码。我们在多个数据集上取得的最先进性能验证了所提方法在少样本分割任务中的有效性。我们的官方代码可在https://github.com/SuhoPark0706/FCP获取。
英文摘要：
We propose Foreground-Covering Prototype Generation and Matching to resolve Few-Shot Segmentation (FSS), which aims to segment target regions in unlabeled query images based on labeled support images. Unlike previous research, which typically estimates target regions in the query using support prototypes and query pixels, we utilize the relationship between support and query prototypes. To achieve this, we utilize two complementary features: SAM Image Encoder features for pixel aggregation and ResNet features for class consistency. Specifically, we construct support and query prototypes with SAM features and distinguish query prototypes of target regions based on ResNet features. For the query prototype construction, we begin by roughly guiding foreground regions within SAM features using the conventional pseudo-mask, then employ iterative cross-attention to aggregate foreground features into learnable tokens. Here, we discover that the cross-attention weights can effectively alternate the conventional pseudo-mask. Therefore, we use the attention-based pseudo-mask to guide ResNet features to focus on the foreground, then infuse the guided ResNet feature into the learnable tokens to generate class-consistent query prototypes. The generation of the support prototype is conducted symmetrically to that of the query one, with the pseudo-mask replaced by the ground-truth mask. Finally, we compare these query prototypes with support ones to generate prompts, which subsequently produce object masks through the SAM Mask Decoder. Our state-of-the-art performances on various datasets validate the effectiveness of the proposed method for FSS. Our official code is available at https://github.com/SuhoPark0706/FCP
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
