基于视觉语言对齐的弱监督3D视觉定位
Weakly-Supervised 3D Visual Grounding based on Visual Language Alignment
AI总结：
针对现有3D视觉定位需大量边界框标注的问题，提出弱监督方法3D-VLA，利用VLMs的语义对齐能力及2D-3D对应关系构建文本-3D对应，在两个数据集上取得与全监督方法相当甚至更优的结果。
摘要：
学习将自然语言查询定位到3D点云中的目标对象或区域对于3D场景理解至关重要。然而，现有的3D视觉定位方法需要为文本查询提供大量边界框标注，获取这些标注既耗时又费力。本文提出3D-VLA，一种基于视觉语言对齐的弱监督3D视觉定位方法。3D-VLA利用当前大规模视觉语言模型（VLMs）在文本与2D图像语义对齐方面的卓越能力，以及2D图像与3D点云之间自然存在的对应关系，在训练过程中无需细粒度边界框标注即可隐式构建文本与3D点云的对应关系。在推理阶段，学习到的文本-3D对应关系将帮助我们在无需2D图像的情况下将文本查询定位到3D目标对象。据我们所知，这是首个通过引入大规模视觉语言模型以弱监督方式研究3D视觉定位的工作，在ReferIt3D和ScanRefer数据集上的大量实验表明，我们的3D-VLA取得了与全监督方法相当甚至更优的结果。
英文摘要：
Learning to ground natural language queries to target objects or regions in 3D point clouds is quite essential for 3D scene understanding. Nevertheless, existing 3D visual grounding approaches require a substantial number of bounding box annotations for text queries, which is time-consuming and labor-intensive to obtain. In this paper, we propose 3D-VLA, a weakly supervised approach for 3D visual grounding based on Visual Linguistic Alignment. Our 3D-VLA exploits the superior ability of current large-scale vision-language models (VLMs) on aligning the semantics between texts and 2D images, as well as the naturally existing correspondences between 2D images and 3D point clouds, and thus implicitly constructs correspondences between texts and 3D point clouds with no need for fine-grained box annotations in the training procedure. During the inference stage, the learned text-3D correspondence will help us ground the text queries to the 3D target objects even without 2D images. To the best of our knowledge, this is the first work to investigate 3D visual grounding in a weakly supervised manner by involving large scale vision-language models, and extensive experiments on ReferIt3D and ScanRefer datasets demonstrate that our 3D-VLA achieves comparable and even superior results over the fully supervised methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
