PanoGrounder: 利用全景场景表示桥接2D和3D，实现基于VLM的3D视觉定位
PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Grounding
AI总结：
提出PanoGrounder框架，通过多模态全景表示与预训练2D VLM结合，实现强泛化能力的3D视觉定位，在ScanRefer和Nr3D上取得最优结果。
摘要：
3D视觉定位（3DVG）是从视觉语言感知到机器人技术的关键桥梁，需要语言理解和3D场景推理。传统的监督模型利用显式3D几何，但由于3D视觉语言数据集的稀缺以及与现代视觉语言模型（VLM）相比推理能力有限，表现出有限的泛化能力。我们提出一个可泛化的3DVG框架PanoGrounder，它将多模态全景表示与预训练的2D VLM相结合，以实现强大的视觉语言推理。全景渲染，辅以3D语义和几何特征，作为2D和3D之间的中间表示，并带来两个主要好处：（i）它们可以直接输入VLM，只需最小的适配；（ii）由于360度视野，它们保留了长距离的物体间关系。我们设计了一个三阶段流程，根据场景布局和几何放置一组紧凑的全景视点，在每个全景渲染上使用VLM定位文本查询，并通过提升将每视图预测融合为单个3D边界框。我们的方法在ScanRefer和Nr3D上取得了最先进的结果，并展示了对未见过的3D数据集和文本改写的强泛化能力。
英文摘要：
3D Visual Grounding (3DVG) is a critical bridge from vision-language perception to robotics, requiring both language understanding and 3D scene reasoning. Traditional supervised models leverage explicit 3D geometry but exhibit limited generalization, owing to the scarcity of 3D vision-language datasets and the limited reasoning capabilities compared to modern vision-language models (VLMs). We propose a generalizable 3DVG framework, PanoGrounder, that couples multi-modal panoramic representation with pretrained 2D VLMs for strong vision-language reasoning. Panoramic renderings, augmented with 3D semantic and geometric features, serve as an intermediate representation between 2D and 3D, and offer two major benefits: (i) they can be directly fed to VLMs with minimal adaptation and (ii) they retain long-range object-to-object relations thanks to their 360-degree field of view. We devise a three-stage pipeline that places a compact set of panoramic viewpoints considering the scene layout and geometry, grounds a text query on each panoramic rendering with a VLM, and fuses per-view predictions into a single 3D bounding box via lifting. Our approach achieves state-of-the-art results on ScanRefer and Nr3D, and demonstrates strong generalization to unseen 3D datasets and text rephrasings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
