推理可移植性：引导MLLMs在RLVR时代的持续学习
Reasoning Portability: Guiding Continual Learning for MLLMs in the RLVR Era
AI总结：
本文提出了一种名为推理可移植性（RP）的机制，通过在持续学习中引入推理层面的约束，改进了多模态大语言模型在RLVR环境下的适应能力，实验表明RDB-CL在提升最后准确率方面优于基线方法。
摘要：
在持续学习中，视觉-语言模型（VLM-CL）旨在不断适应新多模态任务的同时保留先前知识。新兴的将多模态大语言模型（MLLMs）与具有可验证奖励的强化学习（RLVR）相结合的范式，要求一种新的模式来引导持续适应。随着推理能力的进步，现在可以在推理层面施加约束。我们正式化了可移植性，即一个样本级别的度量，用于衡量先前策略行为在新任务中的可重用性，并实证表明推理层面的信号在分布外样本上仍可靠，而答案层面的信号则不然。我们将此形式化为推理可移植性（RP），并提出基于推理的动态平衡持续学习（RDB-CL），该方法根据RP调节RLVR中的每样本Kullback-Leibler正则化：一个紧密的锚点在高RP样本上保留可重用的推理，而低RP样本上的放松锚点则允许探索新的推理路径。实验表明，RDB-CL在提升最后准确率方面优于基线方法，相比 vanilla RLVR 基线提升了+12.0%。
英文摘要：
Vision-Language Models in Continual Learning (VLM-CL) aim to continuously adapt to new multimodal tasks while retaining prior knowledge. The emerging paradigm that couples Multimodal Large Language Models (MLLMs) with Reinforcement Learning with Verifiable Rewards (RLVR) calls for a new pattern to guide continual adaptation. Advances in reasoning capability now make it feasible to impose constraints at the reasoning level. We formalize portability, a sample-level measure of how reusable the previous policy's behavior is on a new task, and empirically show that reasoning-level signals remain reliable on out-of-distribution samples while answer-level signals do not. We instantiate this as Reasoning Portability (RP) and propose Reasoning-based Dynamic Balance Continual Learning (RDB-CL), which modulates the per-sample Kullback-Leibler regularization in RLVR according to RP: a tight anchor preserves reusable reasoning on high-RP samples, while a relaxed anchor on low-RP samples permits exploration of new reasoning pathways. Experiments show that RDB-CL consistently outperforms baselines, improving Last accuracy by +12.0% over the vanilla RLVR baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
