arXiv 2511.031462025-12-28cs.CL

MME-CC：认知能力多模态评估基准

MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive Capacity

Nanjing University（南京大学）

Kaiyuan Zhang, Chenghao Yang, Zhoufutu Wen, Sihang Yuan, Qiuyue Wang, Chaoyi Huang, Guosheng Zhu, He Wang, Huawenyu Lu, Jianing Wen, Jianpeng Jiao, Lishu Luo, L… 展开作者

Kaiyuan Zhang, Chenghao Yang, Zhoufutu Wen, Sihang Yuan, Qiuyue Wang, Chaoyi Huang, Guosheng Zhu, He Wang, Huawenyu Lu, Jianing Wen, Jianpeng Jiao, Lishu Luo, Longxiang Liu, Sijin Wu, Xiaolei Zhu, Xuanliang Zhang, Yu Liu, Ge Zhang, Yi Lin, Guang Shi, Chaoyou Fu, Wenhao Huang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MME-CC提出一个以视觉为基础的多模态评估基准，系统评估大语言模型在空间、几何和知识推理中的认知能力，揭示封闭源模型在总体表现上的优势及空间几何推理的薄弱环节。

摘要：

随着推理模型迅速发展，人类认知中多模态的重要性日益凸显，推动了对以视觉为中心的认知行为的深入研究。然而，现有的多模态基准要么过度强调文本推理，要么未能系统捕捉以视觉为中心的认知行为，导致大语言模型（LLMs）的认知能力评估不足。为此，我们引入MME-CC（多模态认知能力评估基准），这是一个以视觉为基础的基准，将11个代表性推理任务分为三个基本类别：空间、几何和基于知识的推理，并提供对LLMs在这些维度上认知能力的细致分析。基于MME-CC，我们对16个代表性LLMs进行了广泛实验。我们的研究发现，封闭源模型在总体上领先（例如，Gemini-2.5-Pro得分为42.66，而GLM-4.5V得分为30.45），而空间和几何推理仍普遍薄弱（小于等于30%）。我们进一步识别了常见的错误模式，包括方向错误、脆弱的跨视角身份持续性和对反事实指令的不充分遵循，并观察到链式推理通常遵循一个三阶段过程（提取->推理->验证）并严重依赖视觉提取。我们希望这项工作能推动将LLMs的认知能力视为评估和模型设计中的核心要素。

英文摘要：

As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either overemphasize textual reasoning or fall short of systematically capturing vision-centric cognitive behaviors, leaving the cognitive capacity of MLLMs insufficiently assessed. To address this limitation, we introduce MME-CC (Multi-Modal Evaluation benchmark of Cognitive Capacity), a vision-grounded benchmark that organizes 11 representative reasoning tasks into three fundamental categories of visual information: spatial, geometric, and knowledge-based reasoning, and provides fine-grained analyses of MLLMs' cognitive capacity across these dimensions. Based on MME-CC, we conduct extensive experiments over 16 representative MLLMs. Our study reveals that closed-source models currently lead overall (e.g., 42.66 for Gemini-2.5-Pro vs. 30.45 for GLM-4.5V), while spatial and geometric reasoning remain broadly weak (less than or equal to 30%). We further identify common error patterns, including orientation mistakes, fragile cross-view identity persistence, and poor adherence to counterfactual instructions, and observe that Chain-of-Thought typically follows a three-stage process (extract -> reason -> verify) with heavy reliance on visual extraction. We hope this work catalyzes a shift toward treating the cognitive capacity of MLLMs as central to both evaluation and model design.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。