Hyperphantasia: 一个多模态大语言模型心理可视化能力评估基准
Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMs
AI总结：
Hyperphantasia是一个评估多模态大语言模型心理可视化能力的合成基准，通过四个精心设计的谜题揭示了人类与当前模型之间的性能差距，并探索了强化学习在提升视觉模拟能力上的潜力。
摘要：
心理可视化，即内部构建和操作视觉表征的能力，是人类认知的核心组成部分，在涉及推理、预测和抽象的任务中起着关键作用。尽管多模态大语言模型（MLLMs）取得了快速进展，当前的基准测试主要评估被动的视觉感知，对内部构建视觉模式以支持问题解决的更积极能力提供有限的见解。然而，心理可视化是人类的关键认知技能，支持空间导航、预测物理轨迹以及通过想象模拟解决复杂视觉问题的能力。为了填补这一差距，我们引入了Hyperphantasia，一个合成基准，通过四个精心设计的谜题来评估MLLMs的心理可视化能力。每个谜题都是程序生成的，并在三个难度级别上呈现，使模型在增加的复杂性下能够进行受控分析。我们对最新模型的全面评估揭示了人类与MLLMs之间性能的巨大差距。此外，我们探讨了强化学习在提升视觉模拟能力方面的潜力。我们的发现表明，尽管某些模型在识别视觉模式方面表现出部分能力，但稳健的心理可视化仍然是当前MLLMs面临的开放挑战。
英文摘要：
Mental visualization, the ability to construct and manipulate visual representations internally, is a core component of human cognition and plays a vital role in tasks involving reasoning, prediction, and abstraction. Despite the rapid progress of Multimodal Large Language Models (MLLMs), current benchmarks primarily assess passive visual perception, offering limited insight into the more active capability of internally constructing visual patterns to support problem solving. Yet mental visualization is a critical cognitive skill in humans, supporting abilities such as spatial navigation, predicting physical trajectories, and solving complex visual problems through imaginative simulation. To bridge this gap, we introduce Hyperphantasia, a synthetic benchmark designed to evaluate the mental visualization abilities of MLLMs through four carefully constructed puzzles. Each puzzle is procedurally generated and presented at three difficulty levels, enabling controlled analysis of model performance across increasing complexity. Our comprehensive evaluation of state-of-the-art models reveals a substantial gap between the performance of humans and MLLMs. Additionally, we explore the potential of reinforcement learning to improve visual simulation capabilities. Our findings suggest that while some models exhibit partial competence in recognizing visual patterns, robust mental visualization remains an open challenge for current MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
