面向3D视觉定位的统一表示空间
Unified Representation Space for 3D Visual Grounding
AI总结：
针对3D视觉定位中视觉与文本模态存在差距导致定位分类误差的问题，提出UniSpace-3D框架，通过统一表示编码器、多模态对比学习、语言引导查询选择构建统一表示空间，在相关数据集上性能优于基线至少2.24%。
摘要：
3D视觉定位（3DVG）是场景理解中的一项关键任务，旨在根据文本描述识别3D场景中的物体。然而，现有方法依赖于分别预训练的视觉编码器和文本编码器，导致两种模态在空间几何与语义类别方面存在显著差距，这种差异往往会造成物体定位和分类错误。本文提出UniSpace-3D，创新性地为3D视觉定位引入统一表示空间，有效弥合视觉特征与文本特征之间的差距。具体而言，UniSpace-3D包含三项创新设计：其一为统一表示编码器，利用预训练的CLIP模型将视觉特征与文本特征映射到统一表示空间中，有效弥合两种模态间的差距；其二为多模态对比学习模块，可进一步缩小模态差距；其三为语言引导的查询选择模块，利用位置和语义信息识别与文本描述对齐的物体候选点。大量实验表明，在ScanRefer和Nr3D/Sr3D数据集上，UniSpace-3D的性能优于基线模型至少2.24%。代码将在论文录用后公开。
英文摘要：
3D visual grounding (3DVG) is a critical task in scene understanding that aims to identify objects in 3D scenes based on text descriptions. However, existing methods rely on separately pre-trained vision and text encoders, resulting in a significant gap between the two modalities in terms of spatial geometry and semantic categories. This discrepancy often causes errors in object positioning and classification. The paper proposes UniSpace-3D, which innovatively introduces a unified representation space for 3DVG, effectively bridging the gap between visual and textual features. Specifically, UniSpace-3D incorporates three innovative designs: i) a unified representation encoder that leverages the pre-trained CLIP model to map visual and textual features into a unified representation space, effectively bridging the gap between the two modalities; ii) a multi-modal contrastive learning module that further reduces the modality gap; iii) a language-guided query selection module that utilizes the positional and semantic information to identify object candidate points aligned with textual descriptions. Extensive experiments demonstrate that UniSpace-3D outperforms baseline models by at least 2.24% on the ScanRefer and Nr3D/Sr3D datasets. The code will be made available upon acceptance of the paper.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
