理解连续视觉-语言模型中的跨模态贡献：一个理论视角
Understanding Cross-Modal Contributions in Continual Vision-Language Models: A Theoretical Perspective
AI总结：
本文从理论角度分析连续视觉-语言模型中跨模态（视觉-语言）贡献，提出新视角并通过实验验证其有效性，揭示任务顺序和相似性对贡献鲁棒性的影响，提升泛化性能。
摘要：
连续视觉-语言模型通常通过顺序微调来解决；然而，尽管这种范式能够适应新环境（任务），但它本质上以牺牲保持先前获取知识所需的稳定性为代价，强调了先前学习环境（任务）的贡献。虽然现有方法已经充分研究了视觉-语言模型（VLM）中的连续学习和灾难性遗忘，但跨一系列环境的模态特定贡献的理论理解仍然很大程度上未被探索。在本文中，我们提出了一个新的理论视角来理解跨模态（视觉-语言）对连续环境的贡献。我们在大型VLM上实证评估了我们的理论发现，并展示了它们在捕捉环境级跨模态贡献方面的有效性。我们的分析为连续VLM提供了更深入的见解，突出了它们对不同任务顺序和任务间相似性的贡献鲁棒性，以及它们改进的泛化性能。
英文摘要：
Continual vision-language models are commonly addressed through sequential fine-tuning; however, although this paradigm enables adaptation to new environments (tasks), it inherently emphasizes the contribution of previously learned environments (tasks) at the expense of the stability required to preserve previously acquired knowledge. While existing approaches have adequately studied continual learning and catastrophic forgetting in vision-language models (VLMs), the theoretical understanding of modality-specific contributions across a sequence of environments remains largely unexplored. In this paper, we present a new theoretical perspective to understand the cross-modal (vision-language) contributions to consecutive environments. We empirically evaluate our theoretical findings on large VLMs and demonstrate their effectiveness in capturing environment-level cross-modal contributions. Our analysis provides deeper insights into continual VLMs, highlighting their contribution robustness to varying task orders and inter-task similarities, and their improved generalization performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
