开放词汇3D场景理解：基于几何引导的自蒸馏方法
Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation
AI总结：
针对3D-文本数据稀缺问题，提出几何引导自蒸馏（GGSD）方法，利用3D几何先验增强从2D模型蒸馏的知识，并通过自蒸馏提升3D表示，显著超越现有方法。
摘要：
大规模3D-文本配对数据的稀缺对开放词汇3D场景理解构成了巨大挑战，因此，利用互联网规模的2D数据并通过知识蒸馏将其开放词汇能力迁移至3D模型成为一种流行做法。然而，现有的基于蒸馏的3D场景理解方法依赖于2D模型的表示能力，忽视了3D数据所提供的几何先验和固有表示优势的探索。本文提出了一种有效方法，即几何引导自蒸馏（Geometry Guided Self-Distillation, GGSD），以从2D预训练模型中学习更优的3D表示。具体而言，我们首先设计了一个几何引导蒸馏模块，从2D模型中蒸馏知识，然后利用3D几何先验来减轻2D模型中固有的噪声，并增强表示学习过程。由于3D表示的优势，蒸馏后的3D学生模型的性能可以显著超越2D教师模型。这激励我们进一步通过自蒸馏利用3D数据的表示优势。因此，我们提出的GGSD方法在室内和室外基准数据集上的实验表明，其性能大幅超越了现有的开放词汇3D场景理解方法。
英文摘要：
The scarcity of large-scale 3D-text paired data poses a great challenge on open vocabulary 3D scene understanding, and hence it is popular to leverage internet-scale 2D data and transfer their open vocabulary capabilities to 3D models through knowledge distillation. However, the existing distillation-based 3D scene understanding approaches rely on the representation capacity of 2D models, disregarding the exploration of geometric priors and inherent representational advantages offered by 3D data. In this paper, we propose an effective approach, namely Geometry Guided Self-Distillation (GGSD), to learn superior 3D representations from 2D pre-trained models. Specifically, we first design a geometry guided distillation module to distill knowledge from 2D models, and then leverage the 3D geometric priors to alleviate the inherent noise in 2D models and enhance the representation learning process. Due to the advantages of 3D representation, the performance of the distilled 3D student model can significantly surpass that of the 2D teacher model. This motivates us to further leverage the representation advantages of 3D data through self-distillation. As a result, our proposed GGSD approach outperforms the existing open vocabulary 3D scene understanding methods by a large margin, as demonstrated by our experiments on both indoor and outdoor benchmark datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
