MM-IQ：多模态模型中类人抽象与推理能力的基准测试
MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models
AI总结：
本文提出 MM-IQ 基准，系统评估多模态模型的类人抽象与推理能力，揭示现有模型仅略优于随机水平，并发布一个经强化学习训练的高效多模态推理基线模型。
摘要：
IQ 测试一直作为评估人类认知能力的基础方法，其刻意将评估与语言背景、语言熟练程度或领域特定知识解耦，以隔离出抽象与推理方面的核心能力。然而，人工智能研究目前缺乏系统性基准来量化多模态系统中的这些关键认知能力。为填补这一关键空白，我们提出了 MM-IQ，一个综合评估框架，包含一个大规模训练集，内含 4,776 个视觉推理问题，以及 2,710 个经过精心整理的测试条目，覆盖 8 种不同的推理范式。通过对现有开源和专有多模态模型的系统评估，我们的基准揭示了显著的局限性：即使是最先进的架构，其性能也仅略优于随机概率（33.17% 对比 25% 的基线准确率）。这一巨大的性能差距凸显了当前多模态模型在逼近人类基本推理能力方面的不足，强调了需要范式转变式的进展来弥合这一认知鸿沟。此外，受近期大型推理模型兴起的启发，我们还发布了一个多模态推理模型作为基线，该模型通过强化学习并使用可验证奖励函数进行训练，以明显更小的模型规模达到了与最先进模型相当的性能。
英文摘要：
IQ testing has served as a foundational methodology for evaluating human cognitive capabilities, deliberately decoupling assessment from linguistic background, language proficiency, or domain-specific knowledge to isolate core competencies in abstraction and reasoning. Yet, artificial intelligence research currently lacks systematic benchmarks to quantify these critical cognitive capabilities in multimodal systems. To address this crucial gap, we propose MM-IQ, a comprehensive evaluation framework, which comprises a large-scale training set with 4,776 visual reasoning problems and 2,710 meticulously curated test items spanning 8 distinct reasoning paradigms. Through systematic evaluation of existing open-source and proprietary multimodal models, our benchmark reveals striking limitations: even state-of-the-art architectures achieve only marginally superior performance to random chance (33.17% vs. 25% baseline accuracy). This substantial performance chasm highlights the inadequacy of current multimodal models in approximating fundamental human reasoning capacities, underscoring the need for paradigm-shifting advancements to bridge this cognitive divide. Moreover, inspired by the recent surge of large reasoning models, we also release a multimodal reasoning model as the baseline that is trained via reinforcement learning with verifiable reward functions, reaching competitive performance to the state-of-the-art with a notably smaller model size.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
