arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.14641cs.AI
收藏

学习绘制ASCII图提高语言模型的空间推理能力

Learning to Draw ASCII Improves Spatial Reasoning in Language Models

University of California, Santa Cruz(加州大学圣克ruz分校)

Shiyuan Huang, Li Liu, Jincheng He, Leilani H. Gilpin

AI总结:

本文通过Text2Space数据集研究语言模型通过构造视觉布局提升空间推理能力,发现构造与理解训练结合可显著增强空间推理,并在外部基准上验证了其泛化能力。

摘要:

当面临复杂空间问题时,人类会自然地用草图来组织思维,而绘制过程进一步加深理解。本文探讨大型语言模型(LLMs)是否同样适用这一原理:能否通过学习从空间描述中构造显式视觉布局来获得真正的空间理解?我们引入Text2Space数据集,将自然语言描述与真实ASCII网格布局及空间问答对配对,以区分构造空间表示的失败与推理这些表示的失败。我们采用ASCII因其可读性、完全在语言模型的token空间内运作,并以结构化方式编码空间关系。评估显示存在显著的“读写不对称”:LLMs能有效解读ASCII表示但难以从文本生成,这些构造错误会传播到错误答案中。为解决此限制,我们训练模型进行布局构造(文本→ASCII),发现即使不生成ASCII,仅凭文本也能显著提升空间推理能力。结合构造与理解训练进一步放大了这些收益。关键的是,这些改进在三个外部空间推理基准上转移,证明学习构造显式布局能培养出超越训练格式的通用空间理解。

英文摘要:

When faced with complex spatial problems, humans naturally sketch layouts to organize their thinking, and the act of drawing further sharpens their understanding. In this work, we ask whether a similar principle holds for Large Language Models (LLMs): can learning to construct explicit visual layouts from spatial descriptions instill genuine spatial understanding? We introduce Text2Space, a dataset that pairs natural language descriptions with ground-truth ASCII grid layouts and spatial QA pairs, enabling us to separate failures in constructing spatial representations from failures in reasoning over them. We adopt ASCII because it is human-readable, operates entirely within the token space of language models, and encodes spatial relations in a structurally verifiable form. Our evaluation reveals a pronounced "Read-Write Asymmetry": LLMs interpret ASCII representations effectively but struggle to produce them from text, and these construction errors propagate to incorrect answers downstream. To address this limitation, we train models on layout construction (Text$\rightarrow$ASCII) and find that it significantly improves spatial reasoning from text alone, even without producing any ASCII at inference time. Combining construction with comprehension training further amplifies these gains. Crucially, these improvements transfer to three external spatial reasoning benchmarks, demonstrating that, much as sketching sharpens human spatial thinking, learning to construct explicit layouts instills spatial understanding that generalizes beyond the training format.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑