World2Mind: 用于基础模型的方位认知工具包
World2Mind: Cognition Toolkit for Allocentric Spatial Reasoning in Foundation Models
AI总结:
World2Mind通过构建空间认知地图和三阶段推理链,提升基础模型的空间推理能力,使纯文本模型也能实现复杂3D空间推理。
摘要:
实现稳健的空间推理仍然是当前多模态基础模型(MFMs)面临的根本性挑战。现有方法要么通过3D接地数据过度拟合统计捷径,要么仍局限于2D视觉感知,限制了空间推理的准确性和在未见场景中的泛化能力。受生物智能的空间认知映射机制启发,我们提出World2Mind,一种无需训练的空间智能工具包。其核心是利用3D重建和实例分割模型构建结构化空间认知地图,使MFMs能够主动获取感兴趣地标和路线的空间知识。为提供稳健的几何-拓扑先验,World2Mind合成了一个Allocentric-Spatial Tree(AST),利用椭圆参数精确建模地标自上而下的布局。为缓解3D重建的固有不准确性,我们引入包含工具调用评估、模态解耦线索收集和几何-语义交织推理的三阶段推理链。大量实验表明,World2Mind提升了前沿模型,如GPT-5.2的性能,提升幅度为5%~18%。令人惊讶的是,仅依靠AST结构化的文本,纯文本基础模型也能执行复杂的3D空间推理,性能接近高级多模态模型。
英文摘要:
Achieving robust spatial reasoning remains a fundamental challenge for current Multimodal Foundation Models (MFMs). Existing methods either overfit statistical shortcuts via 3D grounding data or remain confined to 2D visual perception, limiting both spatial reasoning accuracy and generalization in unseen scenarios. Inspired by the spatial cognitive mapping mechanisms of biological intelligence, we propose World2Mind, a training-free spatial intelligence toolkit. At its core, World2Mind leverages 3D reconstruction and instance segmentation models to construct structured spatial cognitive maps, empowering MFMs to proactively acquire targeted spatial knowledge regarding interested landmarks and routes of interest. To provide robust geometric-topological priors, World2Mind synthesizes an Allocentric-Spatial Tree (AST) that uses elliptical parameters to model the top-down layout of landmarks accurately. To mitigate the inherent inaccuracies of 3D reconstruction, we introduce a three-stage reasoning chain comprising tool invocation assessment, modality-decoupled cue collection, and geometry-semantics interwoven reasoning. Extensive experiments demonstrate that World2Mind boosts the performance of frontier models, such as GPT-5.2, by 5%~18%. Astonishingly, relying solely on the AST-structured text, purely text-only foundation models can perform complex 3D spatial reasoning, achieving performance approaching that of advanced multimodal models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。