arXiv 2503.106152025-03-18cs.CV

R1-Onevision：通过跨模态形式化推进通用多模态推理

R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization

Zhejiang University（浙江大学）；WeChat Vision, Tencent Inc.（腾讯微信视觉）；Renmin University of China（中国人民大学）

Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, Wei Chen

备注：: Code and Model: https://github.com/Fancy-MLLM/R1-onevision

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出R1-Onevision模型、跨模态形式化流程与数据集，并通过监督微调和强化学习增强多模态推理，同时发布覆盖初中至大学阶段的R1-Onevision-Bench，性能超过GPT-4o等模型。

摘要：

大型语言模型已在复杂文本任务中展现出卓越的推理能力。然而，需要整合视觉与文本信息的多模态推理仍是一项重大挑战。现有视觉语言模型往往难以有效分析和推理视觉内容，导致其在复杂推理任务上的表现欠佳。此外，缺乏全面的基准测试也阻碍了对多模态推理能力的准确评估。本文提出R1-Onevision，这是一种旨在弥合视觉感知与深度推理之间差距的多模态推理模型。为实现这一目标，我们提出一种跨模态推理流程，将图像转换为形式化文本表示，从而实现精确的基于语言的推理。借助该流程，我们构建了R1-Onevision数据集，在多个不同领域提供详细的、逐步的多模态推理标注。我们进一步通过监督微调和强化学习开发R1-Onevision模型，以培养高级推理能力和稳健的泛化能力。为全面评估不同年级的多模态推理表现，我们推出R1-Onevision-Bench，这是一个与人类教育阶段对齐的基准，涵盖从初中到大学及更高阶段的考试。实验结果表明，R1-Onevision达到了最先进性能，在多个具有挑战性的多模态推理基准上优于GPT-4o和Qwen2.5-VL等模型。

英文摘要：

Large Language Models have demonstrated remarkable reasoning capability in complex textual tasks. However, multimodal reasoning, which requires integrating visual and textual information, remains a significant challenge. Existing visual-language models often struggle to effectively analyze and reason visual content, resulting in suboptimal performance on complex reasoning tasks. Moreover, the absence of comprehensive benchmarks hinders the accurate assessment of multimodal reasoning capabilities. In this paper, we introduce R1-Onevision, a multimodal reasoning model designed to bridge the gap between visual perception and deep reasoning. To achieve this, we propose a cross-modal reasoning pipeline that transforms images into formal textural representations, enabling precise language-based reasoning. Leveraging this pipeline, we construct the R1-Onevision dataset which provides detailed, step-by-step multimodal reasoning annotations across diverse domains. We further develop the R1-Onevision model through supervised fine-tuning and reinforcement learning to cultivate advanced reasoning and robust generalization abilities. To comprehensively evaluate multimodal reasoning performance across different grades, we introduce R1-Onevision-Bench, a benchmark aligned with human educational stages, covering exams from junior high school to university and beyond. Experimental results show that R1-Onevision achieves state-of-the-art performance, outperforming models such as GPT-4o and Qwen2.5-VL on multiple challenging multimodal reasoning benchmarks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。