层次间推理：通过层选择性融合恢复视频-语言模型中的时间推理
Lost in Adaptation: Layer-Selective Recovery of Temporal Reasoning in Video-Language Models
AI总结：
本文提出MERIT框架，通过层选择性融合提升视频-语言模型的时间推理能力，同时保持时间感知能力，优于全模型融合和随机层选择。
摘要：
多模态适应使大语言模型具备感知能力，但往往削弱了语言预训练继承的推理能力。在视频-语言模型（VLMs）中，视觉对齐会损害时间推理（TR）能力。本文提出MERIT，一种无需训练的任务驱动模型融合框架，通过在VLM与其配对的文本-only backbone之间搜索层级自注意力融合方案，以提升TR并惩罚时间感知（TP）的退化。在三个代表性VLM和多个挑战性视频基准上，MERIT一致提升了TR，保持或提高了TP，并泛化到四个不同的基准。它还优于全模型融合和随机层选择，表明有效恢复依赖于选择正确的层。干预性遮蔽和帧级归因进一步表明，所选层对推理至关重要，使模型决策偏向于时间相关和因果相关的证据。这些结果表明，有针对性的、感知意识的模型融合可以有效恢复VLM中的时间推理，而无需重新训练。
英文摘要：
Multimodal adaptation can erode temporal reasoning (TR) in video-language models (VLMs), leaving models able to perceive salient events yet unable to infer their temporal and causal structure. We introduce MERIT, a gradient-free framework that repairs this capability through layer-selective model merging. MERIT assigns each self-attention layer a VLM-dominant or LLM-dominant interpolation and uses the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search the resulting combinatorial space under an objective that rewards TR gains while penalizing temporal perception (TP) degradation. Across three VLM families and five video benchmarks, MERIT consistently improves TR while preserving TP; recipes selected on a compact diagnostic set transfer to four unseen benchmarks, with relative gains of up to 27.8%. Interventional masking and frame-level attribution further show that the selected layers are functionally important for reasoning and that MERIT shifts decisions toward temporally distributed, causally relevant evidence. These results establish layer-selective merging as a practical post-hoc mechanism for repairing video temporal reasoning degraded during multimodal adaptation, without learning new parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
