TaCA：使用任务无关兼容适配器升级您的视觉基础模型
TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adapter
AI总结：
本文提出任务无关兼容适配器TaCA，使下游应用无需重训练即可无缝升级视觉基础模型，并在多任务实验中验证其热插拔升级能力。
摘要：
像CLIP这样的视觉基础模型擅长通过自监督方法从大规模数据集中学习特征表示，展现出卓越的迁移学习和泛化能力。基于视觉基础模型的应用日益增多，包括诸如BLIP-2等创新解决方案。这些应用采用预训练的CLIP模型作为上游特征提取器，并训练各种下游模块以完成多样化任务。在需要更新上游基础模型的系统升级场景中，重新训练所有下游模块以适应新基础模型变得必不可少，这既不灵活又效率低下。在本文中，我们引入了一种参数高效且任务无关的适配器，称为TaCA，它能够促进不同基础模型之间的兼容性，同时确保新模型获得增强的性能。TaCA允许下游应用无缝集成性能更优的基础模型，而无需重新训练。我们使用不同规模的模型（参数多达十亿）在视频-文本检索、视频识别和视觉问答等多种任务上对TaCA进行了广泛的实验验证。结果一致证明了TaCA在视觉基础模型热插拔升级方面的涌现能力。代码和模型将在https://github.com/TencentARC/TaCA上提供。
英文摘要：
Visual foundation models like CLIP excel in learning feature representations from extensive datasets through self-supervised methods, demonstrating remarkable transfer learning and generalization capabilities. A growing number of applications based on visual foundation models are emerging, including innovative solutions such as BLIP-2. These applications employ pre-trained CLIP models as upstream feature extractors and train various downstream modules to accomplish diverse tasks. In situations involving system upgrades that require updating the upstream foundation model, it becomes essential to re-train all downstream modules to adapt to the new foundation model, which is inflexible and inefficient. In this paper, we introduce a parameter-efficient and task-agnostic adapter, dubbed TaCA, that facilitates compatibility across distinct foundation models while ensuring enhanced performance for the new models. TaCA allows downstream applications to seamlessly integrate better-performing foundation models without necessitating retraining. We conduct extensive experimental validation of TaCA using different scales of models with up to one billion parameters on various tasks such as video-text retrieval, video recognition, and visual question answering. The results consistently demonstrate the emergent ability of TaCA on hot-plugging upgrades for visual foundation models. Codes and models will be available at https://github.com/TencentARC/TaCA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
