arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.03590cs.CVcs.AI
收藏

大语言模型能否通过像素感知空间智能?基于文本描述的空间智能基准测试

Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions

School of Computer Science & Technology, Beijing Institute of Technology(计算机科学与技术学院,北京理工大学)

Zhongbin Guo, Zhen Yang, Yushan Li, Xinyue Zhang, Wenyu Gao, Jiacheng Wang, Chengzhi Li, Xiangrui Liu, Ping Jian

AI总结:

SiT-Bench通过文本描述评估大语言模型的空间智能,揭示其在全局一致性方面的不足,并表明显式空间推理可提升性能。

摘要:

最近在空间智能(SI)方面的进展主要依赖于视觉-语言模型(VLMs),然而仍有一个关键问题存在:空间理解是来源于视觉编码器还是基本推理架构?受此问题启发,我们引入了SiT-Bench,一个全新的基准测试,用于评估大语言模型(LLMs)在无像素输入情况下空间智能的表现,包含超过3800个专家标注的项目,涵盖五个主要类别和17个子任务,从以自身为中心的导航和视角转换到精细的机器人操作。通过将单视角或多视角场景转换为高保真的、坐标感知的文本描述,我们挑战LLMs进行符号文本推理而非视觉模式匹配。对最先进的(SOTA)LLMs的评估结果表明,虽然模型在局部语义任务上表现出色,但在全局一致性方面仍存在显著的

英文摘要:

Recent advancements in Spatial Intelligence (SI) have predominantly relied on Vision-Language Models (VLMs), yet a critical question remains: does spatial understanding originate from visual encoders or the fundamental reasoning backbone? Inspired by this question, we introduce SiT-Bench, a novel benchmark designed to evaluate the SI performance of Large Language Models (LLMs) without pixel-level input, comprises over 3,800 expert-annotated items across five primary categories and 17 subtasks, ranging from egocentric navigation and perspective transformation to fine-grained robotic manipulation. By converting single/multi-view scenes into high-fidelity, coordinate-aware textual descriptions, we challenge LLMs to perform symbolic textual reasoning rather than visual pattern matching. Evaluation results of state-of-the-art (SOTA) LLMs reveals that while models achieve proficiency in localized semantic tasks, a significant "spatial gap" remains in global consistency. Notably, we find that explicit spatial reasoning significantly boosts performance, suggesting that LLMs possess latent world-modeling potential. Our proposed dataset SiT-Bench serves as a foundational resource to foster the development of spatially-grounded LLM backbones for future VLMs and embodied agents. Our code and benchmark will be released at https://github.com/binisalegend/SiT-Bench .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑