视觉生成通过多模态世界模型解锁类人推理
Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models
AI总结：
本文提出视觉生成在特定任务中优于纯语言推理，通过构建VisWorld-Eval评估套件验证了多模态世界模型提升类人推理的能力。
摘要：
人类通过操控内部世界模型中的概念来构建内部世界模型并进行推理。最近的AI进展，特别是链式思维（CoT）推理，近似于这种人类认知能力，其中世界模型被认为嵌入在大语言模型中。当前系统在形式和抽象领域如数学和编程中实现了专家级性能，主要依赖于语言推理。然而，在需要更丰富表示和先验知识的物理和空间智能领域，它们仍远低于人类。因此，统一多模态模型（UMMs）的出现引发了对更基于互补多模态路径的类人推理的兴趣，尽管其益处仍不明确。从世界模型的角度来看，本文首次系统研究了何时以及如何视觉生成有助于推理。我们的关键观点是视觉优势假设：对于某些任务——特别是那些基于物理世界的基础任务——视觉生成更自然地作为世界模型，而纯粹的语言世界模型则面临由于表示限制或先验知识不足而产生的瓶颈。理论上，我们将内部世界建模作为CoT推理的核心组成部分，并分析不同形式的世界模型之间的区别。经验上，我们识别出需要交错视觉-语言Co T推理的任务，构建了一个新的评估套件VisWorld-Eval。在最先进的UMM上受控实验显示，交错Co T在偏爱视觉世界建模的任务上显著优于纯语言Co T，但在其他任务上没有明显优势。总的来说，这项工作澄清了多模态世界建模在更强大、更类人多模态AI中的潜力。
英文摘要：
Humans construct internal world models and reason by manipulating the concepts within these models. Recent advances in AI, particularly chain-of-thought (CoT) reasoning, approximate such human cognitive abilities, where world models are believed to be embedded within large language models. Expert-level performance in formal and abstract domains such as mathematics and programming has been achieved in current systems by relying predominantly on verbal reasoning. However, they still lag far behind humans in domains like physical and spatial intelligence, which require richer representations and prior knowledge. The emergence of unified multimodal models (UMMs) capable of both verbal and visual generation has therefore sparked interest in more human-like reasoning grounded in complementary multimodal pathways, though their benefits remain unclear. From a world-model perspective, this paper presents the first principled study of when and how visual generation benefits reasoning. Our key position is the visual superiority hypothesis: for certain tasks--particularly those grounded in the physical world--visual generation more naturally serves as world models, whereas purely verbal world models encounter bottlenecks arising from representational limitations or insufficient prior knowledge. Theoretically, we formalize internal world modeling as a core component of CoT reasoning and analyze distinctions among different forms of world models. Empirically, we identify tasks that necessitate interleaved visual-verbal CoT reasoning, constructing a new evaluation suite, VisWorld-Eval. Controlled experiments on a state-of-the-art UMM show that interleaved CoT significantly outperforms purely verbal CoT on tasks that favor visual world modeling, but offers no clear advantage otherwise. Together, this work clarifies the potential of multimodal world modeling for more powerful, human-like multimodal AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
