基于组成性的数据合成用于视觉推理
Composition-Grounded Data Synthesis for Visual Reasoning
AI总结：
COGS通过分解种子问题并重新组合生成合成数据，提升MLLMs在图表和网页等人工图像领域的推理能力。
摘要：
预训练的多模态大语言模型（MLLMs）在多样化的多模态任务中表现出强大的性能，但在标注困难的领域中仍受限于推理能力。在本工作中，我们关注人工图像领域，如图表、渲染文档和网页，这些领域在实践中丰富但缺乏大规模的人类标注推理数据集。我们引入COGS（COmposition-Grounded data Synthesis），一种数据高效的框架，通过少量种子问题为MLLMs赋予先进的推理能力。关键思想是将每个种子问题分解为基本感知和推理因素，然后可以系统地与新图像重新组合，生成大量合成的问答对。每个生成的问题都配以子问题和中间答案，使能够通过因素级过程奖励进行强化学习。在图表推理实验中，COGS显著提高了对未见过问题的性能，最大的提升体现在推理密集和组合性问题上。此外，使用不同种子数据的因子级混合训练在多个数据集之间转移效果更好，表明COGS诱导了可推广的能力而非数据集特定的过拟合。我们进一步证明该框架可以扩展到其他领域，如网页。
英文摘要：
Pretrained multi-modal large language models (MLLMs) demonstrate strong performance on diverse multimodal tasks, but remain limited in reasoning capabilities for domains where annotations are difficult to collect. In this work, we focus on artificial image domains such as charts, rendered documents, and webpages, which are abundant in practice yet lack large-scale human annotated reasoning datasets. We introduce COGS (COmposition-Grounded data Synthesis), a data-efficient framework for equipping MLLMs with advanced reasoning abilities from a small set of seed questions. The key idea is to decompose each seed question into primitive perception and reasoning factors, which can then be systematically recomposed with new images to generate large collections of synthetic question-answer pairs. Each generated question is paired with subquestions and intermediate answers, enabling reinforcement learning with factor-level process rewards. Experiments on chart reasoning show that COGS substantially improves performance on unseen questions, with the largest gains on reasoning-heavy and compositional questions. Moreover, training with a factor-level mixture of different seed data yields better transfer across multiple datasets, suggesting that COGS induces generalizable capabilities rather than dataset-specific overfitting. We further demonstrate that the framework extends beyond charts to other domains such as webpages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
