FocSAM：深入分割一切中的聚焦对象
FocSAM: Delving Deeply into Focused Objects in Segmenting Anything
AI总结：
针对SAM在挑战样本中因无法动态聚焦和交互信息整合不足导致的稳定性问题，提出FocSAM，通过动态窗口多头自注意力和像素级动态ReLU，以约5.6%的CPU推理时间达到最先进分割质量。
摘要：
分割一切模型（SAM）在分割模型中标志着一个显著的里程碑，其突出的特点是强大的零样本能力和处理各种提示的能力。SAM遵循一种流水线，将交互式分割通过大型编码器进行图像预处理，并通过轻量级解码器进行交互式推理，从而确保高效的实时性能。然而，SAM在具有挑战性的样本上，基于此流水线面临稳定性问题。这些问题源于两个主要因素。首先，图像预处理使得SAM无法在交互过程中动态使用图像级放大策略来重新聚焦于目标对象。其次，轻量级解码器难以将交互信息与图像嵌入充分整合。为了解决这两个局限性，我们提出了FocSAM，并在两个关键方面重新设计了流水线。首先，我们提出了动态窗口多头自注意力（Dwin-MSA），以动态地将SAM的图像嵌入重新聚焦于目标对象。Dwin-MSA将注意力计算局部化在目标对象周围，以最小的计算开销增强与对象相关的嵌入。其次，我们提出了像素级动态ReLU（P-DyReLU），以实现从少数初始点击中充分整合交互信息，这些点击对整体分割结果有显著影响。实验上，FocSAM增强了SAM的交互式分割性能，使其在分割质量上达到现有最先进方法的水平，在CPU上仅需该方法约5.6%的推理时间。
英文摘要：
The Segment Anything Model (SAM) marks a notable milestone in segmentation models, highlighted by its robust zero-shot capabilities and ability to handle diverse prompts. SAM follows a pipeline that separates interactive segmentation into image preprocessing through a large encoder and interactive inference via a lightweight decoder, ensuring efficient real-time performance. However, SAM faces stability issues in challenging samples upon this pipeline. These issues arise from two main factors. Firstly, the image preprocessing disables SAM from dynamically using image-level zoom-in strategies to refocus on the target object during interaction. Secondly, the lightweight decoder struggles to sufficiently integrate interactive information with image embeddings. To address these two limitations, we propose FocSAM with a pipeline redesigned on two pivotal aspects. First, we propose Dynamic Window Multi-head Self-Attention (Dwin-MSA) to dynamically refocus SAM's image embeddings on the target object. Dwin-MSA localizes attention computations around the target object, enhancing object-related embeddings with minimal computational overhead. Second, we propose Pixel-wise Dynamic ReLU (P-DyReLU) to enable sufficient integration of interactive information from a few initial clicks that have significant impacts on the overall segmentation results. Experimentally, FocSAM augments SAM's interactive segmentation performance to match the existing state-of-the-art method in segmentation quality, requiring only about 5.6% of this method's inference time on CPUs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
