AI 中文总结
该研究针对异构MLLM跨规模知识迁移机制，提出CDPI线性探针，经多组模型与基准测试发现，融合增益集中于高层推理，且正向迁移多发生在小比率区间。
AI 中文摘要
无需训练的异构多模态大语言模型（MLLM）融合为跨规模能力迁移提供了直接途径，但整体性能提升并未揭示较小模型实际继承了什么。现有研究大多在有限任务集或聚合指标上设计与评估，当评估扩展到更广泛的任务集合时，不同能力能否跨规模迁移仍知之甚少。为探究该问题，我们引入跨规模定向参数注入（CDPI），一种用于分析异构融合期间跨规模知识迁移的简单线性探针。局部理论分析表明，知识迁移选择性首先由能力对共享注入方向的依赖响应决定，二阶曲率效应则约束有效迁移范围。在四组Qwen3-VL模型对和十二个多模态基准测试中，实验揭示了一致的选择性模式：增益集中在推理，尤其是高层推理，而感知性能仍接近原始目标模型。组件级消融进一步显示，高层推理增益主要来自语言模型，比率分析发现正向选择性迁移主要发生在小比率区间。这些发现将跨规模异构MLLM融合重新定义为在狭窄、低干扰区间内的选择性语言侧推理迁移，而非广泛的能力继承。
英文摘要
Training-free fusion of heterogeneous multimodal large language models (MLLMs) provides a direct route for cross-scale capability transfer, yet improvements in aggregate performance do not reveal what a smaller model actually inherits. Existing studies are largely designed and evaluated on limited task sets or aggregate metrics; as evaluation expands to broader task collections, whether different capabilities can transfer across scales remains poorly understood. To investigate this question, we introduce Cross-Scale Directional Parameter Injection (CDPI), a simple linear probe to analyze cross-scale knowledge transfer during heterogeneous fusion. A local theoretical analysis indicates that knowledge transfer selectivity is determined at first order by capability-dependent responses to a shared injection direction, while second-order curvature effects constrain the effective transfer regime. Across four Qwen3-VL model pairs and twelve multimodal benchmarks, our experiments reveal a consistent pattern of selectivity: gains concentrate on reasoning, particularly high-level reasoning, whereas perception performance remains close to that of the original target model. Component-wise ablations further show that high-level reasoning gains arise primarily from the language model, while ratio analysis finds that positive selective transfer occurs mainly in the small-ratio regime. These findings recast cross-scale heterogeneous MLLM fusion as selective language-side reasoning transfer within a narrow, low-interference regime, rather than broad capability inheritance.
Comments17 pages, 6 figures; includes supplementary material. Revised presentation and terminology; results unchanged
