基于三维感知的空间视觉语言建模
Grounded 3D-Aware Spatial Vision-Language Modeling
AI总结：
提出GR3D模型，通过显式2D定位、隐式2D定位和单目3D定位三种互补定位能力，在单一框架内实现空间链式推理，并在定位与非定位空间基准上取得一致提升。
摘要：
我们提出了GR3D，一个空间视觉语言模型，在单一框架内配备了三种互补的定位能力——显式2D定位、隐式2D定位和单目3D定位。GR3D引入了一种隐式定位机制，在生成过程中识别实体提及，并将相应的区域标记插入文本流中，使模型在生成空间链式推理响应时能够即时引用视觉证据。同时，一种区域提示的单目3D定位设计从定位的区域查询中预测相机视图中的3D边界框，并由内在感知归一化和密集几何监督支持。这些定位能力共同使GR3D能够将复杂的空间理解问题分解为定位的2D感知，随后进行3D推理。GR3D在定位和非定位空间基准上均取得了一致的改进，证明了定位作为增强VLM空间理解的有效归纳偏差。这些定位能力共同增强了超越定位任务本身的通用空间理解。
英文摘要：
We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
