BALD-SAM:基于分歧的交互分割主动提示
BALD-SAM: Disagreement-based Active Prompting in Interactive Segmentation
AI总结:
BALD-SAM通过引入贝叶斯主动学习与分歧方法,提升交互分割的自动化和准确性,实现跨领域高性能表现。
摘要:
段落任何模型(SAM)通过空间提示革新了交互分割。尽管现有工作主要关注在各种设置中自动化提示,现实中的标注流程涉及迭代细化,标注人员观察模型输出并战略性地放置提示以解决歧义。当前流程通常依赖标注人员对预测掩码质量的视觉评估。我们推测,一种有原则的自动化交互提示方法是使用模型衍生的标准来识别下一个提示最信息丰富的区域。在本工作中,我们建立了主动提示:一种空间主动学习方法,其中图像内的位置构成一个未标记的池子,提示作为查询以优先选择信息丰富的区域,提高每次交互的效用。我们进一步提出了BALD-SAM:一种将贝叶斯主动学习通过分歧(BALD)适应到空间提示选择的框架,通过量化知识不确定性来实现。为此,我们冻结了整个模型,并仅将贝叶斯不确定性建模应用于一个小型学习预测头,使对大规模多百万参数基础模型的不可行不确定性估计变得可行。在涵盖自然、医疗、水下和地震领域的16个数据集中,BALD-SAM展示了强大的跨领域性能,在16个基准中的14个中排名第一或第二。我们通过涵盖3个SAM骨干网络和35个拉普拉斯后验配置的全面消融套件验证了这些收益,共计38种不同的消融设置。除了强大的平均性能外,BALD-SAM超越了人类提示,并在某些类别中甚至超越了Oracle提示,同时在最终分割质量上始终优于一次基线,在薄和结构复杂的对象上尤其如此。
英文摘要:
The Segment Anything Model (SAM) has revolutionized interactive segmentation through spatial prompting. While existing work primarily focuses on automating prompts in various settings, real-world annotation workflows involve iterative refinement where annotators observe model outputs and strategically place prompts to resolve ambiguities. Current pipelines typically rely on the annotator's visual assessment of the predicted mask quality. We postulate that a principled approach for automated interactive prompting is to use a model-derived criterion to identify the most informative region for the next prompt. In this work, we establish active prompting: a spatial active learning approach where locations within images constitute an unlabeled pool and prompts serve as queries to prioritize information-rich regions, increasing the utility of each interaction. We further present BALD-SAM: a principled framework adapting Bayesian Active Learning by Disagreement (BALD) to spatial prompt selection by quantifying epistemic uncertainty. To do so, we freeze the entire model and apply Bayesian uncertainty modeling only to a small learned prediction head, making intractable uncertainty estimation practical for large multi-million parameter foundation models. Across 16 datasets spanning natural, medical, underwater, and seismic domains, BALD-SAM demonstrates strong cross-domain performance, ranking first or second on 14 of 16 benchmarks. We validate these gains through a comprehensive ablation suite covering 3 SAM backbones and 35 Laplace posterior configurations, amounting to 38 distinct ablation settings. Beyond strong average performance, BALD-SAM surpasses human prompting and, in several categories, even oracle prompting, while consistently outperforming one-shot baselines in final segmentation quality, particularly on thin and structurally complex objects.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。