arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.19306cs.CV
收藏

用于少样本分割的对称联合学习支持-查询原型

Symmetrical Joint Learning Support-query Prototypes for Few-shot Segmentation

School of Computer Science, Nanjing University of Posts and Telecommunications(南京邮电大学计算机学院);Singapore University of Technology and Design(新加坡科技设计大学);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);University of California at Riverside(加州大学河滨分校)

Qun Li, Baoquan Sun, Fu Xiao, Yonggang Qi, Bir Bhanu

AI总结:

提出 Sym-Net 框架,通过视觉-文本对齐、无参数先验掩码生成和超相关模块,对称联合学习支持与查询原型,有效解决少样本分割中的类内变化问题,显著提升分割性能。

摘要:

我们提出了 Sym-Net,一个用于少样本分割(FSS)的新框架,通过以对称方式联合学习查询和支持原型来解决类内变化的关键问题。不同于以往仅通过将查询特征与支持原型匹配来生成查询原型的方法(这是一种对少样本支持样本的偏置学习),Sym-Net 对查询和支持原型采用均衡的对称学习方法,确保学习过程不偏向任何一方(支持集或查询集)。Sym-Net 的主要模块之一是基于视觉-文本对齐的原型聚合模块,它不仅仅是查询引导的原型细化,而是从支持样本和查询样本中联合学习,这使得模型有利于处理类内差异,并能更好地泛化到新的未见类别。具体而言,设计了一个无参数先验掩码生成模块,通过使用不同大小的滑动窗口和自激活核来抑制错误的背景匹配,从而准确定位查询对象的局部和全局区域。此外,为了解决原型学习过程中空间池化导致的信息丢失问题,集成了一个自上而下的超相关模块,以捕获支持图像和查询图像之间的多尺度空间关系。该方法通过实施协同优化的硬三元组挖掘策略进一步联合优化。实验结果表明,所提出的 Sym-Net 优于现有最先进的模型,这表明以对称方式为 FSS 联合学习支持-查询原型为在有限标注数据下增强分割性能提供了一个有前景的方向。

英文摘要:

We propose Sym-Net, a novel framework for Few-Shot Segmentation (FSS) that addresses the critical issue of intra-class variation by jointly learning both query and support prototypes in a symmetrical manner. Unlike previous methods that generate query prototypes solely by matching query features to support prototypes, which is a form of bias learning towards the few-shot support samples, Sym-Net leverages a balanced symmetrical learning approach for both query and support prototypes, ensuring that the learning process does not favor one set (support or query) over the other. One of main modules of Sym-Net is the visual-text alignment-based prototype aggregation module, which is not just query-guided prototype refinement, it is a jointly learning from both support and query samples, which makes the model beneficial for handling intra-class discrepancies and allows it to generalize better to new, unseen classes. Specifically, a parameter-free prior mask generation module is designed to accurately localize both local and global regions of the query object by using sliding windows of different sizes and a self-activation kernel to suppress incorrect background matches. Additionally, to address the information loss caused by spatial pooling during prototype learning, a top-down hyper-correlation module is integrated to capture multi-scale spatial relationships between support and query images. This approach is further jointly optimized by implementing a co-optimized hard triplet mining strategy. Experimental results show that the proposed Sym-Net outperforms state-of-the-art models, which demonstrates that jointly learning support-query prototypes in a symmetrical manner for FSS offers a promising direction to enhance segmentation performance with limited annotated data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑