R-4B：通过双模式退火与强化学习激励多模态大语言模型的通用自动思考能力
R-4B: Incentivizing General-Purpose Auto-Thinking Capability in MLLMs via Bi-Mode Annealing and Reinforce Learning
AI总结：
针对多模态大模型思考过程对简单问题冗余的问题，提出R-4B自动思考模型，通过双模式退火与BPO算法自适应决策是否启动思考，在25个基准上达SOTA，性能媲美更大模型且成本更低。
摘要：
具备逐步思考能力的多模态大语言模型（MLLMs）在复杂推理任务上已展现出优异性能，但对于无需复杂推理即可解决的简单问题，该思考过程存在冗余。为解决这一低效问题，我们提出R-4B，一种自动思考MLLM，可根据问题复杂度自适应决定何时启动思考。R-4B的核心思路是通过双模式退火让模型同时具备思考与非思考能力，并采用双模式策略优化（BPO）提升模型判断是否激活思考流程的准确率。具体而言，我们首先在精心策划的、覆盖多主题且包含思考与非思考模式样本的数据集上训练模型；随后在改进的GRPO框架下进行第二阶段训练，强制策略模型对每个输入查询生成两种模式的回复。实验结果显示，R-4B在25个具有挑战性的基准测试上取得了当前最优性能，在多数任务上优于Qwen2.5-VL-7B，且在推理密集型基准上以更低计算成本达到了与Kimi-VL-A3B-Thinking-2506（16B）等更大模型相当的性能。
英文摘要：
Multimodal Large Language Models (MLLMs) equipped with step-by-step thinking capabilities have demonstrated remarkable performance on complex reasoning problems. However, this thinking process is redundant for simple problems solvable without complex reasoning. To address this inefficiency, we propose R-4B, an auto-thinking MLLM, which can adaptively decide when to think based on problem complexity. The central idea of R-4B is to empower the model with both thinking and non-thinking capabilities using bi-mode annealing, and apply Bi-mode Policy Optimization (BPO) to improve the model's accuracy in determining whether to activate the thinking process. Specifically, we first train the model on a carefully curated dataset spanning various topics, which contains samples from both thinking and non-thinking modes. Then it undergoes a second phase of training under an improved GRPO framework, where the policy model is forced to generate responses from both modes for each input query. Experimental results show that R-4B achieves state-of-the-art performance across 25 challenging benchmarks. It outperforms Qwen2.5-VL-7B in most tasks and achieves performance comparable to larger models such as Kimi-VL-A3B-Thinking-2506 (16B) on reasoning-intensive benchmarks with lower computational cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
