GoodSAM++：利用Segment Anything Model弥合全景语义分割的域差距与容量差距
GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentation
AI总结：
本研究提出无标注全景语义分割框架GoodSAM++，通过引入教师助理结合SAM生成伪语义图，搭配畸变校正与多级知识适配模块，以轻量学生模型实现了媲美SOTA的性能与强泛化性。
摘要：
本文提出GoodSAM++，这是一种新颖的框架，利用SAM（即Segment Anything Model，作为教师模型）强大的零样本实例分割能力，在无需任何标注数据的情况下训练紧凑的全景语义分割模型（即学生模型）。GoodSAM++解决了两大关键挑战：1）SAM无法提供语义标签，以及全景图像固有的畸变问题；2）SAM与学生模型之间显著的容量差距。GoodSAM++的核心创新思路是引入教师助理（TA）为SAM提供语义信息，与SAM结合得到可靠的伪语义图，同时弥合域差距与容量差距。为实现这一目标，我们首先提出畸变感知校正（DARv2）模块以解决域差距问题，该模块有效缓解全景图像中的物体形变与畸变问题，生成伪语义图。随后我们引入多级知识适配（MKA）模块，将来自教师助理的语义信息和伪语义图高效迁移到我们的紧凑学生模型中，解决显著的容量差距。我们在室外和室内基准数据集上开展了大量实验，结果表明GoodSAM++相比当前最优（SOTA）的域自适应方法取得了显著的性能提升。此外，多种开放世界场景验证了GoodSAM++的泛化能力。最后值得一提的是，我们最轻量化的学生模型仅含370万参数，性能却可与SOTA模型相媲美。
英文摘要：
This paper presents GoodSAM++, a novel framework utilizing the powerful zero-shot instance segmentation capability of SAM (i.e., teacher) to learn a compact panoramic semantic segmentation model, i.e., student, without requiring any labeled data. GoodSAM++ addresses two critical challenges: 1) SAM's inability to provide semantic labels and inherent distortion problems of panoramic images; 2) the significant capacity disparity between SAM and the student. The `out-of-the-box' insight of GoodSAM++ is to introduce a teacher assistant (TA) to provide semantic information for SAM, integrated with SAM to obtain reliable pseudo semantic maps to bridge both domain and capacity gaps. To make this possible, we first propose a Distortion-Aware Rectification (DARv2) module to address the domain gap. It effectively mitigates the object deformation and distortion problem in panoramic images to obtain pseudo semantic maps. We then introduce a Multi-level Knowledge Adaptation (MKA) module to efficiently transfer the semantic information from the TA and pseudo semantic maps to our compact student model, addressing the significant capacity gap. We conduct extensive experiments on both outdoor and indoor benchmark datasets, showing that our GoodSAM++ achieves a remarkable performance improvement over the state-of-the-art (SOTA) domain adaptation methods. Moreover, diverse open-world scenarios demonstrate the generalization capacity of our GoodSAM++. Last but not least, our most lightweight student model achieves comparable performance to the SOTA models with only 3.7 million parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
