arXiv 2607.008812026-07-02cs.CV

OmniView-Space: 通过多视角空间映射增强空间推理

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（厦门大学多媒体可信感知与高效计算教育部重点实验室）；Tencent Youtu Lab（腾讯优图实验室）；Beijing Institute of Technology（北京理工大学）

Xudong Li, Mengdan Zhang, Peixian Chen, Jiaxi Tan, Zihao Huang, Jingyuan Zheng, Yan Zhang, Xiawu Zheng, Xing Sun, Rongrong Ji

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出OmniView-Space框架，通过多视角空间映射、工具引导的自我中心推理和认知图蒸馏，解决多模态大模型在多步空间推理中的参考框架动态重锚问题，在单/多图像基准上达到最优性能。

摘要：

空间智能对多模态大语言模型（MLLMs）来说仍然是一个持续的挑战，因为它需要超越基本物体识别的连贯空间场景表示。现有方法通常通过文本推理或3D重建来构建此类表示。然而，它们在多步推理中常常出错，特别是在需要根据复杂查询动态地将证据重新锚定到特定的相机、物体或方向为中心的参考框架时。为了解决这个问题，我们提出了OmniView-Space，一个旨在通过多模态自我中心证据维持空间一致性的框架。我们的方法包含三个核心组件：（1）多视角空间映射（MPSM），它将重建的几何重新锚定到与查询对齐的视觉认知图和文本空间图中；（2）工具引导的自我中心推理，一种交错策略，训练模型主动选择查询所需的自我锚点并请求相应的MPSM证据；（3）认知图蒸馏，利用MPSM生成的轨迹和自我框架奖励来训练模型使用自生成的认知图进行推理。在单图像和多图像空间推理基准上的实验表明，OmniView-Space达到了最先进的性能。此外，蒸馏后的模型在减少对外部几何管线依赖的同时保持了这一性能。

英文摘要：

Spatial intelligence remains a persistent challenge for Multimodal Large Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual reasoning or 3D reconstruction. However, they often falter during multi-step reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。