SAPNet++: 基于语义和空间感知的点提示实例分割
SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness
AI总结：
SAPNet++通过整合点距离指导和框挖掘策略，提升点提示实例分割的精度和边界处理能力。
摘要：
单点标注在视觉任务中越来越突出，用于降低标注成本。然而，它挑战了需要高精度的任务，如点提示实例分割（PPIS）任务，其目标是使用单点提示训练分割网络以估计精确的掩码。由于点标注的限制，粒度模糊和边界不确定性导致区分不同细节层次（例如整体对象与部分）的困难，以及精确界定对象边界挑战。先前的工作通常继承了掩码生成与提案选择的范式以实现PPIS。然而，提案选择仅依赖类别信息，无法解决不同粒度的模糊性。此外，掩码生成器只能提供有限的离散解决方案，往往偏离实际的掩码，特别是在边界处。为了解决这些问题，我们提出了语义感知的点提示实例分割网络（SAPNet）。它集成了点距离指导和框挖掘策略，以解决由点的粒度模糊性引起的组和局部问题。此外，我们还在提案中加入了完整性分数，以增加空间粒度感知，增强提案选择中的多实例学习（MIL）称为S-MIL。多级亲和力细化传达像素和语义线索，在掩码细化过程中缩小边界不确定性。这些模块最终构成了SAPNet++，缓解了点提示的粒度模糊和边界不确定性，并显著提高了分割性能。在四个具有挑战性的数据集上进行了广泛的实验，验证了我们方法的有效性，突显了其在PPIS中的潜力。
英文摘要：
Single-point annotation is increasingly prominent in visual tasks for labeling cost reduction. However, it challenges tasks requiring high precision, such as the point-prompted instance segmentation (PPIS) task, which aims to estimate precise masks using single-point prompts to train a segmentation network. Due to the constraints of point annotations, granularity ambiguity and boundary uncertainty arise the difficulty distinguishing between different levels of detail (eg. whole object vs. parts) and the challenge of precisely delineating object boundaries. Previous works have usually inherited the paradigm of mask generation along with proposal selection to achieve PPIS. However, proposal selection relies solely on category information, failing to resolve the ambiguity of different granularity. Furthermore, mask generators offer only finite discrete solutions that often deviate from actual masks, particularly at boundaries. To address these issues, we propose the Semantic-Aware Point-Prompted Instance Segmentation Network (SAPNet). It integrates Point Distance Guidance and Box Mining Strategy to tackle group and local issues caused by the point's granularity ambiguity. Additionally, we incorporate completeness scores within proposals to add spatial granularity awareness, enhancing multiple instance learning (MIL) in proposal selection termed S-MIL. The Multi-level Affinity Refinement conveys pixel and semantic clues, narrowing boundary uncertainty during mask refinement. These modules culminate in SAPNet++, mitigating point prompt's granularity ambiguity and boundary uncertainty and significantly improving segmentation performance. Extensive experiments on four challenging datasets validate the effectiveness of our methods, highlighting the potential to advance PPIS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
