视觉图竞技场：评估视觉与多模态大语言模型的视觉概念化能力
Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models
AI总结：
本文提出视觉图竞技场（VGA）数据集，通过六项图任务评估视觉与多模态大语言模型的视觉概念化能力，发现现有模型在同构检测等表示不变推理上远逊于人类，并揭示其伪智能模式匹配问题。
摘要：
近年来，多模态大语言模型的进展推动了视觉问答的突破。然而，一个关键差距仍然存在，即“概念化”——识别并推理同一概念的能力，尽管其视觉形式发生变化，这是人类推理的一项基本能力。为应对这一挑战，我们引入了视觉图竞技场（Visual Graph Arena，VGA），这是一个包含六项基于图的任务的数据集，旨在评估和提升人工智能系统的视觉抽象能力。VGA 使用多样的图布局（例如 Kamada-Kawai 布局与平面布局）来测试独立于视觉形式的推理能力。对最先进的视觉模型和多模态大语言模型的实验揭示了一个显著的分化：人类在所有任务上取得了接近完美的准确率，而模型在同构检测上完全失败，在路径/环任务中仅取得有限成功。我们进一步识别出行为异常，表明其属于伪智能模式匹配而非真正的理解。这些发现凸显了当前人工智能模型在视觉理解方面的根本局限。通过隔离表示不变推理这一挑战，VGA 提供了一个框架，以推动人工智能视觉模型向类人概念化方向发展。视觉图竞技场可在 https://vga.csail.mit.edu/ 获取。
英文摘要：
Recent advancements in multimodal large language models have driven breakthroughs in visual question answering. Yet, a critical gap persists, `conceptualization'-the ability to recognize and reason about the same concept despite variations in visual form, a basic ability of human reasoning. To address this challenge, we introduce the Visual Graph Arena (VGA), a dataset featuring six graph-based tasks designed to evaluate and improve AI systems' capacity for visual abstraction. VGA uses diverse graph layouts (e.g., Kamada-Kawai vs. planar) to test reasoning independent of visual form. Experiments with state-of-the-art vision models and multimodal LLMs reveal a striking divide: humans achieved near-perfect accuracy across tasks, while models totally failed on isomorphism detection and showed limited success in path/cycle tasks. We further identify behavioral anomalies suggesting pseudo-intelligent pattern matching rather than genuine understanding. These findings underscore fundamental limitations in current AI models for visual understanding. By isolating the challenge of representation-invariant reasoning, the VGA provides a framework to drive progress toward human-like conceptualization in AI visual models. The Visual Graph Arena is available at: \href{https://vga.csail.mit.edu/}{vga.csail.mit.edu}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
