大语言模型的心智之眼：思维可视化激发大语言模型的空间推理能力
Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
AI总结：
本文受人类“心智之眼”认知能力启发，提出思维可视化（VoT）提示方法，通过可视化大语言模型的推理轨迹来激发其空间推理能力，并在多跳空间推理任务上显著超越现有多模态大语言模型。
摘要：
大语言模型（LLMs）在语言理解和各类推理任务中已展现出令人瞩目的性能。然而，它们在空间推理这一人类认知的关键方面的能力，仍相对未被充分探索。人类拥有一种非凡的能力，可以通过一种被称为“心智之眼”的过程，对看不见的物体和动作生成心理图像，从而实现对不可见世界的想象。受这一认知能力的启发，我们提出了思维可视化（Visualization-of-Thought，VoT）提示方法。VoT旨在通过将大语言模型的推理轨迹可视化来激发其空间推理能力，从而引导后续的推理步骤。我们将VoT应用于多跳空间推理任务，包括自然语言导航、视觉导航以及二维网格世界中的视觉拼块。实验结果表明，VoT显著增强了大语言模型的空间推理能力。值得注意的是，VoT在这些任务上的表现优于现有多模态大语言模型（MLLMs）。虽然VoT在大语言模型上表现出奇地有效，但生成心理图像以促进空间推理的能力与心智之眼过程相似，这表明其在多模态大语言模型中具有潜在的可行性。数据集和代码可在 https://microsoft.github.io/visualization-of-thought 获取。
英文摘要：
Large language models (LLMs) have exhibited impressive performance in language comprehension and various reasoning tasks. However, their abilities in spatial reasoning, a crucial aspect of human cognition, remain relatively unexplored. Human possess a remarkable ability to create mental images of unseen objects and actions through a process known as the Mind's Eye, enabling the imagination of the unseen world. Inspired by this cognitive capacity, we propose Visualization-of-Thought (VoT) prompting. VoT aims to elicit spatial reasoning of LLMs by visualizing their reasoning traces, thereby guiding subsequent reasoning steps. We employed VoT for multi-hop spatial reasoning tasks, including natural language navigation, visual navigation, and visual tiling in 2D grid worlds. Experimental results demonstrated that VoT significantly enhances the spatial reasoning abilities of LLMs. Notably, VoT outperformed existing multimodal large language models (MLLMs) in these tasks. While VoT works surprisingly well on LLMs, the ability to generate mental images to facilitate spatial reasoning resembles the mind's eye process, suggesting its potential viability in MLLMs. Please find the dataset and codes at https://microsoft.github.io/visualization-of-thought
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
