arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.00537cs.CV
收藏

提升 Segment Anything 模型以泛化视觉非显著场景

Boosting Segment Anything Model to Generalize Visually Non-Salient Scenarios

Unmanned System Research Institute at Northwestern Polytechnical University(西北工业大学无人系统研究院);School of Electronic, Electrical, and Communication Engineering, University of Chinese Academic of Sciences(中国科学院大学电子电气与通信工程学院);Max Planck Institute for Informatics (MPI-INF)(马克斯·普朗克研究所(信息研究所));University of Science and Technology of China(中国科学技术大学)

Guangqian Guo, Pengfei Chen, Yong Guo, Huafeng Chen, Boqiang Zhang, Shan Gao

AI总结:

本文提出VNS-SAM,通过改进Segment Anything模型以提升对视觉非显著场景的分割性能和泛化能力。

摘要:

Segment Anything 模型(SAM),以其卓越的零样本分割能力而闻名,在社区中引起了广泛关注。然而,当处理我们称为视觉非显著场景时,其性能受到挑战,这些场景中前景与背景之间的对比度较低。在这些情况下,现有方法往往无法准确捕捉轮廓,并且无法产生有前途的分割结果。在本文中,我们提出了视觉非显著 SAM(VNS-SAM),旨在增强 SAM 对视觉非显著场景的感知能力,同时保持其原始的零样本泛化能力。我们通过两种设计有效地利用 SAM 的低级特征来实现这一目标:Mask-Edge Token Interactive decoder 和 Non-Salient Feature Mining 模块。这些设计帮助 SAM 解码器更深入地理解非显著特征,仅需微小的参数增加和计算需求。VNS-SAM 的额外参数可以在 4 小时内进行优化,证明了其可行性和实用性。在数据方面,我们建立了 VNS-SEG,一个用于各种 VNS 场景的统一数据集,包含超过 35,000 张图像,与之前的单任务适应不同。它旨在使模型学习更稳健的 VNS 特征,并全面评估模型在 VNS 场景中的分割性能和泛化能力。在各种 VNS 分割任务上的广泛实验显示了 VNS-SAM 的优越性能,特别是在零样本设置下,突显了其在广泛现实应用中的潜力。代码和数据集可在 https://guangqian-guo.github.io/VNS-SAM 上公开获取。

英文摘要:

Segment Anything Model (SAM), known for its remarkable zero-shot segmentation capabilities, has garnered significant attention in the community. Nevertheless, its performance is challenged when dealing with what we refer to as visually non-salient scenarios, where there is low contrast between the foreground and background. In these cases, existing methods often cannot capture accurate contours and fail to produce promising segmentation results. In this paper, we propose Visually Non-Salient SAM (VNS-SAM), aiming to enhance SAM's perception of visually non-salient scenarios while preserving its original zero-shot generalizability. We achieve this by effectively exploiting SAM's low-level features through two designs: Mask-Edge Token Interactive decoder and Non-Salient Feature Mining module. These designs help the SAM decoder gain a deeper understanding of non-salient characteristics with only marginal parameter increments and computational requirements. The additional parameters of VNS-SAM can be optimized within 4 hours, demonstrating its feasibility and practicality. In terms of data, we established VNS-SEG, a unified dataset for various VNS scenarios, with more than 35K images, in contrast to previous single-task adaptations. It is designed to make the model learn more robust VNS features and comprehensively benchmark the model's segmentation performance and generalizability on VNS scenarios. Extensive experiments across various VNS segmentation tasks demonstrate the superior performance of VNS-SAM, particularly under zero-shot settings, highlighting its potential for broad real-world applications. Codes and datasets are publicly available at https://guangqian-guo.github.io/VNS-SAM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑