通过主动3D场景探索增强MLLM空间理解以实现多视角推理
Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning
AI总结：
本文提出一种无需训练的框架，通过显式3D重建机制提升MLLM的空间理解能力，通过主动探索生成新视角，优于专门空间模型和通用MLLM。
摘要：
尽管多模态大语言模型取得了显著进展，但其依赖2D视觉先验仍难以处理复杂的3D空间推理。现有方法通常通过计算昂贵的后训练过程或缺乏显式几何理解的刚性工具调用机制来缓解这一限制。为解决这些挑战，我们提出了一种训练自由的框架，引入基于显式3D重建的视觉链式思维机制。该框架首先利用MLLM引导的关键字提取和掩码生成在多个粒度上重建高保真的3D网格。随后，该框架利用外部知识库迭代计算最优相机外参，并合成新视角，从而模拟人类视角取向。大量实验表明，所提出的方法显著提升了空间理解能力。具体而言，该框架在3DSRBench和Rel3D等主要基准上优于专门的空间模型和通用MLLM，包括GPT-5.2和Gemini-2.5-Flash。
英文摘要：
Although Multimodal Large Language Models have achieved remarkable progress, they still struggle with complex 3D spatial reasoning due to the reliance on 2D visual priors. Existing approaches typically mitigate this limitation either through computationally expensive post-training procedures on limited 3D datasets or through rigid tool-calling mechanisms that lack explicit geometric understanding and viewpoint flexibility. To address these challenges, we propose a \textit{training-free} framework that introduces a Visual Chain-of-Thought mechanism grounded in explicit 3D reconstruction. The proposed pipeline first reconstructs a high-fidelity 3D mesh from a single image using MLLM-guided keyword extraction and mask generation at multiple granularities. Subsequently, the framework leverages an external knowledge base to iteratively compute optimal camera extrinsic parameters and synthesize novel views, thereby emulating human perspective-taking. Extensive experiments demonstrate that the proposed approach significantly enhances spatial comprehension. Specifically, the framework outperforms specialized spatial models and general-purpose MLLMs, including \textit{GPT-5.2} and \textit{Gemini-2.5-Flash}, on major benchmarks such as 3DSRBench and Rel3D.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
