EasyARC：在真实视觉推理上评估视觉语言模型
EasyARC: Evaluating Vision Language Models on True Visual Reasoning
AI总结：
针对现有多模态基准缺乏真实视觉推理的问题，本文提出程序化生成的EasyARC视觉语言基准，支持多步推理与渐进难度，测试前沿模型并分析失效模式，为视觉语言推理评估设立新标准。
摘要：
基于语言推理模型的最新进展，我们探索融合视觉与文本的多模态推理。现有多模态基准主要测试视觉提取结合文本推理的能力，缺乏涉及视觉与语言间更复杂交互的真实视觉推理。受ARC挑战启发，我们推出EasyARC——一个需要多图像、多步推理及自我纠错的视觉语言基准。EasyARC通过程序化生成，完全可验证且可扩展，非常适合强化学习（RL）流程。生成器包含渐进式难度等级，可实现跨任务类型与复杂度的结构化评估。我们对前沿视觉语言模型进行基准测试并分析其失效模式，认为EasyARC为评估视觉语言模型的真实推理能力及测试时扩展能力设立了新标准。我们已开源该基准数据集与评估代码。
英文摘要：
Building on recent advances in language-based reasoning models, we explore multimodal reasoning that integrates vision and text. Existing multimodal benchmarks primarily test visual extraction combined with text-based reasoning, lacking true visual reasoning with more complex interactions between vision and language. Inspired by the ARC challenge, we introduce EasyARC, a vision-language benchmark requiring multi-image, multi-step reasoning, and self-correction. EasyARC is procedurally generated, fully verifiable, and scalable, making it ideal for reinforcement learning (RL) pipelines. The generators incorporate progressive difficulty levels, enabling structured evaluation across task types and complexities. We benchmark state-of-the-art vision-language models and analyze their failure modes. We argue that EasyARC sets a new standard for evaluating true reasoning and test-time scaling capabilities in vision-language models. We open-source our benchmark dataset and evaluation code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
