arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06333cs.CV
收藏

驯服野 SAM3:一个概念库用于开放词汇分割

Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation

Department of Electrical and Computer Engineering, Sungkyunkwan University(Sungkyunkwan 大学电气与计算机工程系);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Computer Science and Engineering, University of Electronic Science and Technology of China(中国电子科技大学计算机科学与工程学院)

Gensheng Pei, Xiruo Jiang, Yazhou Yao, Xiangbo Shu, Fumin Shen, Byeungwoo Jeon

AI总结:

ConceptBank通过构建目标统计的概念库,提升SAM3在数据和概念漂移下的鲁棒性和效率。

摘要:

最近引入的SAM3通过可提示的概念分割革命化了开放词汇分割(OVS)。然而,这种对预定义概念的依赖使模型容易受到威胁:当视觉分布变化(数据漂移)或条件标签分布演变(概念漂移)时,视觉证据与提示之间的对齐会破裂。在本文中,我们提出了ConceptBank,一个无参数的校准框架,用于在飞行中恢复这种对齐。我们不依赖静态提示,而是从目标统计中构建特定数据集的概念库。我们的方法(i)通过类别的视觉原型锚定目标域证据,(ii)挖掘代表性支持以抑制异常值在数据漂移下,(iii)融合候选概念以纠正概念漂移。我们证明ConceptBank能够有效地使SAM3适应分布漂移,包括具有挑战性的自然场景和遥感场景,建立了OVS中稳健性和效率的新基准。代码和模型可在https://github.com/pgsmall/ConceptBank获取。

英文摘要:

The recent introduction of \texttt{SAM3} has revolutionized Open-Vocabulary Segmentation (OVS) through \textit{promptable concept segmentation}, which grounds pixel predictions in flexible concept prompts. However, this reliance on pre-defined concepts makes the model vulnerable: when visual distributions shift (\textit{data drift}) or conditional label distributions evolve (\textit{concept drift}) in the target domain, the alignment between visual evidence and prompts breaks down. In this work, we present \textsc{ConceptBank}, a parameter-free calibration framework to restore this alignment on the fly. Instead of adhering to static prompts, we construct a dataset-specific concept bank from the target statistics. Our approach (\textit{i}) anchors target-domain evidence via class-wise visual prototypes, (\textit{ii}) mines representative supports to suppress outliers under data drift, and (\textit{iii}) fuses candidate concepts to rectify concept drift. We demonstrate that \textsc{ConceptBank} effectively adapts \texttt{SAM3} to distribution drifts, including challenging natural-scene and remote-sensing scenarios, establishing a new baseline for robustness and efficiency in OVS. Code and model are available at https://github.com/pgsmall/ConceptBank.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑