VisuRiddles：细粒度感知是多模态大语言模型在抽象视觉推理中的主要瓶颈
VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning
AI总结：
本文提出AVR基准VisuRiddles与自动合成框架PRS，通过细粒度感知描述监督中间推理，验证视觉感知是MLLMs抽象推理的主要瓶颈并显著提升性能。
摘要：
多模态大语言模型（MLLMs）的最新进展显著提升了其在许多推理任务中的表现。然而，抽象视觉推理（AVR）仍然是一项关键挑战，主要原因在于模型在感知抽象图形方面存在局限。为解决这一问题，我们研究了当前MLLMs中的瓶颈，并合成训练数据以改进其抽象视觉感知。首先，我们提出了用于AVR的基准VisuRiddles，其任务经过精心构建，旨在评估模型在五个核心维度和两个高级推理类别上的推理能力。其次，我们引入了Perceptual Riddle Synthesizer（PRS，感知谜题合成器），这是一个用于生成带有细粒度感知描述的谜题的自动化框架。PRS不仅为抽象图形生成有价值的训练数据，还提供细粒度感知描述，关键在于能够对中间推理阶段进行监督，从而提高训练效果和模型可解释性。我们在VisuRiddles上的大量实验结果从经验上验证了细粒度视觉感知是主要瓶颈，并且我们的合成框架显著提升了当代MLLMs在这些具有挑战性的任务上的性能。我们的代码和数据集将在https://github.com/yh-hust/VisuRiddles发布。
英文摘要：
Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks. However, Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics. To tackle this issue, we investigate the bottlenecks in current MLLMs and synthesize training data to improve their abstract visual perception. First, we propose VisuRiddles, a benchmark for AVR, featuring tasks meticulously constructed to assess models' reasoning capacities across five core dimensions and two high-level reasoning categories. Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions. PRS not only generates valuable training data for abstract graphics but also provides fine-grained perceptual description, crucially allowing for supervision over intermediate reasoning stages and thereby improving both training efficacy and model interpretability. Our extensive experimental results on VisuRiddles empirically validate that fine-grained visual perception is the principal bottleneck and our synthesis framework markedly enhances the performance of contemporary MLLMs on these challenging tasks. Our code and dataset will be released at https://github.com/yh-hust/VisuRiddles
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
