DSM：为3D视觉定位构建多样语义地图
DSM: Constructing a Diverse Semantic Map for 3D Visual Grounding
AI总结：
提出DSM框架，融合VLM语义与几何模型构建多样语义地图，并基于此提出DSM-Grounding范式，在ScanRefer和语义分割任务上取得最优性能，并成功应用于真实机器人导航与抓取。
摘要：
有效的场景表示对于表征的视觉定位能力至关重要，然而现有的3D视觉定位方法常常受到限制。它们要么只关注几何和视觉线索，要么像传统的3D场景图那样，缺乏复杂推理所需的多维属性。为弥合这一差距，我们引入了多样语义地图（DSM）框架，这是一种新颖的场景表示框架，用一系列从VLM（视觉语言模型）派生的语义（包括外观、物理属性和可供性）来丰富稳健的几何模型。DSM首先通过时间滑动窗口内融合多视角观测进行在线构建，创建一个持久且全面的世界模型。在此基础之上，我们提出了DSM-Grounding，这是一种新的范式，将定位从自由形式的VLM查询转变为对语义丰富地图的结构化推理过程，显著提高了准确性和可解释性。大量评估验证了我们方法的优越性。在ScanRefer基准上，DSM-Grounding在IoU@0.5下达到了最先进的59.06%总体准确率，超越其他方法10%。在语义分割中，我们的DSM达到了67.93%的F-mIoU，优于所有基线，包括特权基线。此外，在物理机器人上成功部署用于复杂导航和抓取任务，证实了该框架在现实场景中的实际效用。
英文摘要：
Effective scene representation is critical for the visual grounding ability of representations, yet existing methods for 3D Visual Grounding are often constrained. They either only focus on geometric and visual cues, or, like traditional 3D scene graphs, lack the multi-dimensional attributes needed for complex reasoning. To bridge this gap, we introduce the Diverse Semantic Map (DSM) framework, a novel scene representation framework that enriches robust geometric models with a spectrum of VLM-derived semantics, including appearance, physical properties, and affordances. The DSM is first constructed online by fusing multi-view observations within a temporal sliding window, creating a persistent and comprehensive world model. Building on this foundation, we propose DSM-Grounding, a new paradigm that shifts grounding from free-form VLM queries to a structured reasoning process over the semantic-rich map, markedly improving accuracy and interpretability. Extensive evaluations validate our approach's superiority. On the ScanRefer benchmark, DSM-Grounding achieves a state-of-the-art 59.06% overall accuracy of IoU@0.5, surpassing others by 10%. In semantic segmentation, our DSM attains a 67.93% F-mIoU, outperforming all baselines, including privileged ones. Furthermore, successful deployment on physical robots for complex navigation and grasping tasks confirms the framework's practical utility in real-world scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
