MentisOculi: 揭示心智图像推理的局限性
MentisOculi: Revealing the Limits of Reasoning with Mental Imagery
AI总结：
提出MentisOculi基准，通过多步推理问题测试前沿模型利用视觉表示辅助推理的能力，发现视觉策略普遍无法提升性能，且统一多模态模型存在生成错误累积和无法利用真实可视化的问题。
摘要：
前沿模型正从仅摄入视觉信息的多模态大语言模型（MLLMs）过渡到能够原生交错生成的统一多模态模型（UMMs）。这一转变激发了将中间可视化作为推理辅助的兴趣，类似于人类的心智图像。这一想法的核心是能够以目标导向的方式形成、维护和操作视觉表示。为了评估和探究这一能力，我们开发了MentisOculi，这是一个程序化的、分层的多步推理问题套件，适用于视觉解决方案，旨在挑战前沿模型。评估从潜在令牌到显式生成图像的视觉策略，我们发现它们通常无法提升性能。对UMMs的分析特别揭示了一个关键限制：虽然它们拥有解决任务的文本推理能力，并且有时能生成正确的视觉内容，但它们遭受复合生成错误，并且无法利用甚至真实的可视化。我们的发现表明，尽管视觉思维具有内在吸引力，但尚未有益于模型推理。MentisOculi为分析和弥合不同模型家族之间的这一差距建立了必要的基础。
英文摘要：
Frontier models are transitioning from multimodal large language models (MLLMs) that merely ingest visual information to unified multimodal models (UMMs) capable of native interleaved generation. This shift has sparked interest in using intermediate visualizations as a reasoning aid, akin to human mental imagery. Central to this idea is the ability to form, maintain, and manipulate visual representations in a goal-oriented manner. To evaluate and probe this capability, we develop MentisOculi, a procedural, stratified suite of multi-step reasoning problems amenable to visual solution, tuned to challenge frontier models. Evaluating visual strategies ranging from latent tokens to explicit generated imagery, we find they generally fail to improve performance. Analysis of UMMs specifically exposes a critical limitation: While they possess the textual reasoning capacity to solve a task and can sometimes generate correct visuals, they suffer from compounding generation errors and fail to leverage even ground-truth visualizations. Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model reasoning. MentisOculi establishes the necessary foundation to analyze and close this gap across diverse model families.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
