SAM-MI：一种注入掩码的框架，用于通过SAM增强开放词汇语义分割
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
AI总结：
SAM-MI通过引入掩码注入框架，结合SAM解决开放词汇语义分割中的过度分割和标签结合问题，提升分割性能和效率。
摘要：
开放词汇语义分割（OVSS）旨在对和识别对象进行普遍分割。在大量高质量分割数据上训练的分割任何模型（SAM）已显示出显著的普遍分割能力，为OVSS提供了有价值的支持。尽管以前的方法在利用SAM进行OVSS方面取得了进展，但仍存在一些挑战：（1）SAM倾向于过度分割；（2）固定掩码和标签之间的硬结合。本文介绍了一种新的掩码注入框架SAM-MI，有效整合SAM与OVSS模型以解决这些挑战。最初，SAM-MI使用文本引导的稀疏点提示器为SAM采样稀疏提示，而不是以前的密集网格状提示，从而显著加速了掩码生成过程。该框架随后引入浅层掩码聚合（SMAgg）以合并部分掩码以缓解SAM的过度分割问题。最后，解耦掩码注入（DMI）将SAM生成的掩码分别用于低频和高频指导，而不是直接与标签结合。在多个基准上的广泛实验验证了SAM-MI的优越性。值得注意的是，所提出的方法在MESS基准上相对于Grounded-SAM的mIoU实现了16.7%的相对提升，并伴有1.6倍的速度提升。我们希望SAM-MI可以作为一种替代方法，有效为OVSS模型配备SAM。
英文摘要：
Open-vocabulary semantic segmentation (OVSS) aims to segment and recognize objects universally. Trained on extensive high-quality segmentation data, the segment anything model (SAM) has demonstrated remarkable universal segmentation capabilities, offering valuable support for OVSS. Although previous methods have made progress in leveraging SAM for OVSS, there are still some challenges: (1) SAM's tendency to over-segment and (2) hard combinations between fixed masks and labels. This paper introduces a novel mask-injected framework, SAM-MI, which effectively integrates SAM with OVSS models to address these challenges. Initially, SAM-MI employs a Text-guided Sparse Point Prompter to sample sparse prompts for SAM instead of previous dense grid-like prompts, thus significantly accelerating the mask generation process. The framework then introduces Shallow Mask Aggregation (SMAgg) to merge partial masks to mitigate the SAM's over-segmentation issue. Finally, Decoupled Mask Injection (DMI) incorporates SAM-generated masks for guidance at low-frequency and high-frequency separately, rather than directly combining them with labels. Extensive experiments on multiple benchmarks validate the superiority of SAM-MI. Notably, the proposed method achieves a 16.7% relative improvement in mIoU over Grounded-SAM on the MESS benchmark, along with a 1.6$\times$ speedup. We hope SAM-MI can serve as an alternative methodology to effectively equip the OVSS model with SAM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
