arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.03002cs.AI
收藏

SpatialText: 一种纯文本的认知基准,用于大型语言模型中的空间理解

SpatialText: A Pure-Text Cognitive Benchmark for Spatial Understanding in Large Language Models

Zhejiang University(浙江大学);School of Software Technology, Zhejiang University(软件技术学院,浙江大学)

Peiyao Jiang, Zequn Qin, Xi Li

AI总结:

SpatialText通过双源方法为大型语言模型提供纯文本空间认知基准,揭示其在空间推理中的系统性缺陷。

摘要:

真正的空间推理依赖于构建和操作连贯的内部空间表示的能力,通常被概念化为心理模型,而不是仅仅处理表面的语言关联。尽管大型语言模型在各种领域表现出高级能力,但现有的基准测试未能将这种内在的空间认知与统计语言启发式方法分离。此外,多模态评估经常将真正的空间推理与视觉感知混为一谈。为了系统地调查模型是否构建了灵活的空间心理模型,我们引入了SpatialText,一种理论驱动的诊断框架。与其仅仅作为数据集的功能不同,SpatialText通过双源方法隔离基于文本的空间推理。它整合了人工标注的真实3D室内环境描述,这些描述捕捉了自然的歧义、视角变化和功能关系,与代码生成的逻辑精确场景相结合,旨在探测正式的空间推理和知识边界。在最先进的模型上的系统评估揭示了基本的表示限制。尽管模型在检索显式空间事实和在全局、 allocentric坐标系统中操作方面表现出熟练,但在 egocentric 视角转换和局部参考框架推理方面却存在关键失败。这些系统性错误提供了强有力的证据,表明当前模型严重依赖于语言共现启发式方法,而不是构建连贯、可验证的内部空间表示。因此,SpatialText成为诊断人工智能空间智能认知边界的严格工具。

英文摘要:

Genuine spatial reasoning relies on the capacity to construct and manipulate coherent internal spatial representations, often conceptualized as mental models, rather than merely processing surface linguistic associations. While large language models exhibit advanced capabilities across various domains, existing benchmarks fail to isolate this intrinsic spatial cognition from statistical language heuristics. Furthermore, multimodal evaluations frequently conflate genuine spatial reasoning with visual perception. To systematically investigate whether models construct flexible spatial mental models, we introduce SpatialText, a theory-driven diagnostic framework. Rather than functioning simply as a dataset, SpatialText isolates text-based spatial reasoning through a dual-source methodology. It integrates human-annotated descriptions of real 3D indoor environments, which capture natural ambiguities, perspective shifts, and functional relations, with code-generated, logically precise scenes designed to probe formal spatial deduction and epistemic boundaries. Systematic evaluation across state-of-the-art models reveals fundamental representational limitations. Although models demonstrate proficiency in retrieving explicit spatial facts and operating within global, allocentric coordinate systems, they exhibit critical failures in egocentric perspective transformation and local reference frame reasoning. These systematic errors provide strong evidence that current models rely heavily on linguistic co-occurrence heuristics rather than constructing coherent, verifiable internal spatial representations. SpatialText thus serves as a rigorous instrument for diagnosing the cognitive boundaries of artificial spatial intelligence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑