OpenOcc：基于占据表示的开放词汇3D场景重建
OpenOcc: Open Vocabulary 3D Scene Reconstruction via Occupancy Representation
AI总结：
本文提出OpenOcc框架，结合神经辐射场与占据表示，并通过体渲染蒸馏开放词汇模型，实现零样本3D场景重建与理解。
摘要：
3D重建已广泛应用于移动机器人的自主导航领域。然而，以往研究只能提供基本几何结构，不具备开放世界场景理解能力，限制了人机交互和视觉导航等高级任务。此外，传统3D场景理解方法依赖昂贵的带标注3D数据集，以监督方式为单一任务训练模型。因此，兼具零样本场景理解的几何重建，即开放词汇3D理解与重建，对移动机器人的未来发展至关重要。本文提出OpenOcc，这是一种利用神经辐射场统一3D场景重建与开放词汇理解的新颖框架。我们使用占据表示对场景几何结构建模，并通过体渲染将预训练的开放词汇模型蒸馏到3D语言场中，以实现零样本推理。此外，本文提出一种新颖的语义感知置信度传播（SCP）方法，以缓解蒸馏特征中测量不一致所导致的语言场表示退化问题。实验结果表明，我们的方法在3D场景理解任务中取得了具有竞争力的性能，尤其是在小型和长尾对象方面。
英文摘要：
3D reconstruction has been widely used in autonomous navigation fields of mobile robotics. However, the former research can only provide the basic geometry structure without the capability of open-world scene understanding, limiting advanced tasks like human interaction and visual navigation. Moreover, traditional 3D scene understanding approaches rely on expensive labeled 3D datasets to train a model for a single task with supervision. Thus, geometric reconstruction with zero-shot scene understanding i.e. Open vocabulary 3D Understanding and Reconstruction, is crucial for the future development of mobile robots. In this paper, we propose OpenOcc, a novel framework unifying the 3D scene reconstruction and open vocabulary understanding with neural radiance fields. We model the geometric structure of the scene with occupancy representation and distill the pre-trained open vocabulary model into a 3D language field via volume rendering for zero-shot inference. Furthermore, a novel semantic-aware confidence propagation (SCP) method has been proposed to relieve the issue of language field representation degeneracy caused by inconsistent measurements in distilled features. Experimental results show that our approach achieves competitive performance in 3D scene understanding tasks, especially for small and long-tail objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
