我们的基准测试足够吗？多模态大语言模型持续学习分析
Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs
AI总结：
本文质疑MR-LoRA方法对MLLM路由器的依赖，提出无训练无重放的简单原型路由方法RePRo，并指出共享专家无益，揭示MLLM-CL基准的任务高度可分离和固定顺序导致评估偏差，从而提出新基准设计。
摘要：
持续适应对于部署在不断变化领域中的多模态大语言模型（MLLMs）至关重要，但最先进的MR-LoRA方法高度依赖于一个假设：基于MLLM的路由器对于处理复杂的多模态输入是必要的。本文在MLLM-CL基准上重新审视这一主张，并提出两个论点。\n\n\textbf{第一}，路由不需要MLLM：一种简单的无训练、无重放的原型路由方法（\textsc{RePRo}），使用冻结的预训练特征和任务原型，以远低于MR-LoRA的计算成本匹配其基于MLLM的路由器。\n\n\textbf{第二}，共享专家并不能改善MLLM的持续学习，尽管它们在理论上有吸引力。我们表明，这些发现源于MLLM-CL的两个结构限制：（1）其任务在表示空间中\textbf{高度可分离}，（2）其固定的任务顺序使得结论\textbf{对单一课程敏感}，而不是在不同持续学习轨迹上鲁棒。因此，该基准主要奖励孤立学习而非真正的持续迁移。这激励了未来持续MLLM学习基准的新设计，包括重叠的任务流形、多个任务顺序、细粒度的领域偏移以及奖励前向迁移和保留的评估协议。
英文摘要：
Continual adaptation is essential for multimodal large language models (MLLMs) deployed across evolving domains, but the state-of-the-art MR-LoRA method highly relies on the assumption that a MLLM-based router is necessary to process complex multimodal inputs. This paper revisits this claim on the MLLM-CL benchmark and argues for two claims. \textbf{First}, routing does not require an MLLM: a simple training-free, replay-free ptotypical routing method (\textsc{RePRo}), uses frozen pretrained features and task prototypes to match the MLLM-based router of MR-LoRA at far lower computational cost. \textbf{Second}, shared experts do not improve continual learning for MLLMs, despite their theoretical appeal. We show that these findings arise from two structural limitations of MLLM-CL: (1) its tasks are \textbf{highly separable} in representation space, and (2) its fixed task order makes conclusions \textbf{sensitive to a single curriculum} rather than robust across diverse continual-learning trajectories. As a result, the benchmark primarily rewards learning in isolation rather than genuine continual transfer. This motivates a new design for future benchmarks of continual MLLM learning, with overlapping task manifolds, multiple task orders, fine-grained domain shifts, and evaluation protocols that reward forward transfer as well as retention.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
