对比语言-彩色点图预训练用于统一3D场景理解
RGB-Pointmap Pretraining for Unified 3D Scene Understanding
AI总结：
提出UniScene3D，一种基于Transformer的编码器，通过多视图彩色点图联合建模外观和几何，并引入跨视图几何对齐和接地视图对齐，在少样本和任务特定微调中达到SOTA。
摘要：
通过与对比语言-图像预训练（CLIP）对齐来预训练3D编码器已成为学习3D场景理解可泛化表示的有前景方向。本文提出UniScene3D，一种基于Transformer的编码器，从多视图彩色点图中学习统一场景表示，联合建模图像外观和几何。为了鲁棒的彩色点图表示学习，我们引入了新颖的跨视图几何对齐和接地视图对齐，以强制跨视图几何和语义一致性。在视点接地、场景检索、场景类型分类和3D VQA上的大量少样本和任务特定微调评估展示了我们的最先进性能。这些结果突显了我们的方法在统一3D场景理解中的有效性。
英文摘要：
Pretraining 3D encoders through alignment with Contrastive Language-Image Pre-training (CLIP) has emerged as a promising direction for learning generalizable representations for 3D scene understanding. In this paper, we propose UniScene3D, a transformer-based framework that learns unified 3D scene representations from multi-view RGB-Pointmap inputs by leveraging the priors of a pretrained 2D foundation model. For robust RGB-Pointmap representation learning, we introduce cross-view geometric alignment and grounded view alignment to enforce geometric and semantic consistency across views. Extensive low-shot and task-specific fine-tuning on viewpoint grounding, scene retrieval, scene classification, and 3D visual question answering demonstrates state-of-the-art performance. These results establish UniScene3D as an effective framework for unified 3D scene understanding. Project page: https://yebulabula.github.io/UniScene3D/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
