HoneyBee: 用于视觉-语言推理器的数据食谱
HoneyBee: Data Recipes for Vision-Language Reasoners
AI总结：
本文提出HoneyBee数据集，通过数据整理方法提升视觉-语言推理能力，发现上下文来源策略和数据干预显著提升性能，并提出测试时缩放策略以降低解码成本。
摘要：
近年来，视觉-语言模型（VLMs）在推理任务中表现出色，但其训练数据构建原则仍不明确。本文引入了多种数据整理方法，并通过严格控制训练和评估设置研究其对推理能力的影响。我们分析了上下文来源（图像和问题对）的影响，实施了有针对性的数据干预，并探索了图像、问题和推理链（CoT）的扩展。研究发现，上下文来源策略显著影响VLM性能，辅助信号如图像描述和文本推理的加入带来了显著提升，同时扩展所有数据维度（如每个图像的唯一问题和每个图像-问题对的唯一CoT）能持续提高推理能力。基于这些发现，我们引入了HoneyBee，一个包含250万示例的大规模CoT推理数据集，包含35万个图像-问题对。训练HoneyBee的VLM在模型大小上均优于现有最佳模型。例如，一个3B参数的HoneyBee训练VLM在MathVerse上优于SOTA模型和基础模型分别7.8%和24.8%。此外，我们提出了一种测试时缩放策略，使解码成本降低73%而不牺牲准确性。本文为视觉-语言推理数据集整理研究提供了改进策略。数据可在https://huggingface.co/datasets/facebook/HoneyBee获取。
英文摘要：
Recent advances in vision-language models (VLMs) have made them highly effective at reasoning tasks. However, the principles underlying the construction of performant VL reasoning training datasets remain poorly understood. In this work, we introduce several data curation approaches and study their impacts on VL reasoning capabilities by carefully controlling training and evaluation setups. We analyze the effects of context (image and question pair) sources, implement targeted data interventions, and explore scaling up images, questions, and chain-of-thought (CoT) solutions. Our findings reveal that (a) context source strategies significantly affect VLM performance, (b) interventions such as auxiliary signals from image captions and the inclusion of text-only reasoning yield substantial gains, and (c) scaling all data dimensions (e.g., unique questions per image and unique CoTs per image-question pair) consistently improves reasoning capability. Motivated by these insights, we introduce HoneyBee, a large-scale, high-quality CoT reasoning dataset with 2.5M examples consisting 350K image-question pairs. VLMs trained with HoneyBee outperform state-of-the-art models across model sizes. For instance, a HoneyBee-trained VLM with 3B parameters outperforms the SOTA model and the base model by 7.8% and 24.8%, respectively, on MathVerse. Furthermore, we propose a test-time scaling strategy that reduces decoding cost by 73% without sacrificing accuracy. Overall, this work presents improved strategies for VL reasoning dataset curation research. Data is available at https://huggingface.co/datasets/facebook/HoneyBee.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
