SplatReasoner：通过新颖视图合成增强具身推理与基础能力
SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis
AI总结：
研究针对视觉语言模型应用于具身场景理解受固定视角限制的问题，提出SplatReasoner框架，利用3D高斯点云将新颖视图合成引入推理过程，经实验验证该方法能提升具身推理和3D基础能力。
摘要：
视觉语言模型（VLMs）在图像和视频上展现出强大推理能力，但应用于具身场景理解时，常受限于情景RGB-D记忆中的固定视角。这些观察可能因遮挡、物体截断、视野受限或视图组合不佳而无法捕捉与查询相关的证据。我们提出SplatReasoner框架，通过利用3D高斯点云（3DGS）将新颖视图合成引入VLM推理过程。给定关于3D场景的用户查询，SplatReasoner检索相关观察并合成查询条件视角，以揭示回答查询和在3D中定位所指实体所需的视觉证据。实验表明，查询条件新颖视图合成在固定视角记忆和语言嵌入3DGS基线之上，提升了具身推理和3D基础能力。
英文摘要：
Vision-Language Models (VLMs) have demonstrated strong reasoning capabilities over images and videos, yet their application to embodied scene understanding often constrained by the fixed viewpoints stored in episodic RGB-D memories. These observations may fail to capture query-relevant evidence due to occlusions, object truncation, restricted fields of view, or suboptimal view composition. We present SplatReasoner, a framework that introduces novel view synthesis into the VLM reasoning process by leveraging 3D Gaussian Splatting (3DGS). Given a user query about a 3D scene, SplatReasoner retrieves relevant observations and synthesizes query-conditioned viewpoints that reveal the visual evidence needed to answer the query and ground the referred entities in 3D. Experiments show that query-conditioned novel view synthesis improves both embodied reasoning and 3D grounding over fixed-view memory and language-embedded 3DGS baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
