Where，而非 What：促使视频 LLM 学习用于 3D Grounding 的几何因果关系
Where, Not What: Compelling Video LLMs to Learn Geometric Causality for 3D-Grounding
AI总结：
针对视频 LLM 在 3D grounding 中过度依赖 2D 语义的问题，本文提出 W2R2 训练框架，通过解耦 What/Where 表示并抑制 2D 捷径，在 ScanRefer 与 ScanQA 上显著提升定位精度和鲁棒性。
摘要：
多模态 3D grounding 已在 Vision-Language Models（VLM，视觉-语言模型）中引起广泛关注，用于提升复杂环境中的空间推理能力。然而，这些模型存在严重的“2D 语义偏差”：其成因是过度依赖 2D 图像特征进行粗略定位，在很大程度上忽视 3D 几何输入，并导致次优的融合性能。本文提出一种名为 What-Where Representation Re-Forming（W2R2）的新型训练框架，通过解耦表示学习和有针对性的捷径抑制来解决该问题。我们的方法从根本上重塑模型的内部空间：将 2D 特征指定为用于“What”识别的语义信标，将 3D 特征指定为用于“Where”定位的空间锚点，从而在不修改推理架构的情况下实现精确 3D grounding。关键组件包括一个双目标损失函数：其中 Alignment Loss 使用适配的交叉熵监督融合预测以实现多模态协同，Pseudo-Label Loss 则通过基于间隔的机制惩罚过于有效的 2D 主导伪输出。在 ScanRefer 和 ScanQA 上进行的实验证明了 W2R2 的有效性，其在定位精度和鲁棒性上均取得显著提升，尤其是在杂乱的户外场景中。
英文摘要：
Multimodal 3D grounding has garnered considerable interest in Vision-Language Models (VLMs) \cite{yin2025spatial} for advancing spatial reasoning in complex environments. However, these models suffer from a severe "2D semantic bias" that arises from over-reliance on 2D image features for coarse localization, largely disregarding 3D geometric inputs and resulting in suboptimal fusion performance. In this paper, we propose a novel training framework called What-Where Representation Re-Forming (W2R2) to tackle this issue via disentangled representation learning and targeted shortcut suppression. Our approach fundamentally reshapes the model's internal space by designating 2D features as semantic beacons for "What" identification and 3D features as spatial anchors for "Where" localization, enabling precise 3D grounding without modifying inference architecture. Key components include a dual-objective loss function with an Alignment Loss that supervises fused predictions using adapted cross-entropy for multimodal synergy, and a Pseudo-Label Loss that penalizes overly effective 2D-dominant pseudo-outputs via a margin-based mechanism. Experiments conducted on ScanRefer and ScanQA demonstrate the effectiveness of W2R2, with significant gains in localization accuracy and robustness, particularly in cluttered outdoor scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
