面向3D场景理解的基于对象对比学习的视觉-语言预训练
Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding
AI总结：
提出3DVLP框架，通过对象级IoU引导检测损失及跨对比、自对比任务，有效提取通用3D视觉语言嵌入，在多个3D视觉语言下游任务上取得优异性能。
摘要：
近年来，视觉语言预训练框架在自然语言处理和计算机视觉领域取得了显著进展，在各种下游任务上实现了出色的性能提升。然而，当扩展到点云数据时，现有工作主要集中在构建特定任务模型，未能提取出泛化良好的通用3D视觉语言嵌入。我们仔细研究了语义3D场景理解中的三个常见任务，并得出了开发预训练模型的关键见解。受这些观察的启发，我们提出了一种视觉语言预训练框架3DVLP（基于对象对比学习的3D视觉语言预训练），该框架能灵活迁移到3D视觉语言下游任务上。3DVLP将视觉定位作为代理任务，并引入对象级IoU引导检测（OID）损失，以在场景中获取高质量的提议。此外，我们设计了对象级跨对比对齐（OCC）任务和对象级自对比学习（OSC）任务，分别将对象与描述进行对齐，并区分场景中的不同对象。大量实验验证了3DVLP在三个3D视觉语言任务上的优异性能，体现了其在语义3D场景理解方面的优越性。
英文摘要：
In recent years, vision language pre-training frameworks have made significant progress in natural language processing and computer vision, achieving remarkable performance improvement on various downstream tasks. However, when extended to point cloud data, existing works mainly focus on building task-specific models, and fail to extract universal 3D vision-language embedding that generalize well. We carefully investigate three common tasks in semantic 3D scene understanding, and derive key insights into the development of a pre-training model. Motivated by these observations, we propose a vision-language pre-training framework 3DVLP (3D vision-language pre-training with object contrastive learning), which transfers flexibly on 3D vision-language downstream tasks. 3DVLP takes visual grounding as the proxy task and introduces Object-level IoU-guided Detection (OID) loss to obtain high-quality proposals in the scene. Moreover, we design Object-level Cross-Contrastive alignment (OCC) task and Object-level Self-Contrastive learning (OSC) task to align the objects with descriptions and distinguish different objects in the scene, respectively. Extensive experiments verify the excellent performance of 3DVLP on three 3D vision-language tasks, reflecting its superiority in semantic 3D scene understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
