选择与蒸馏：面向视觉-语言模型持续学习的选择性双教师知识迁移
Select and Distill: Selective Dual-Teacher Knowledge Transfer for Continual Learning on Vision-Language Models
AI总结：
提出选择性双教师知识迁移框架，利用微调和原始VLM作为双教师，通过特征差异进行选择性蒸馏，缓解持续学习中的灾难性遗忘并保持零样本能力。
摘要：
大规模视觉-语言模型（VLMs）在未见领域数据上展现出强大的零样本泛化能力。然而，将预训练的VLMs适应一系列下游任务往往会导致先前学习知识的遗忘以及零样本分类性能的下降。为解决这一问题，我们提出了一种独特的“选择性双教师知识迁移”框架，该框架利用最近微调的VLMs和原始预训练的VLMs作为双教师，分别保留先前学习的知识和零样本能力。仅借助无标签的参考数据集，我们提出的框架通过测量双教师VLM之间的特征差异，执行一种选择性知识蒸馏机制。因此，我们的选择性双教师知识蒸馏缓解了先前学习知识的灾难性遗忘，同时保留了预训练VLMs的零样本能力。在基准数据集上的大量实验表明，我们的框架在防止灾难性遗忘和零样本退化方面优于最先进的持续学习方法。项目页面：https://chuyu.org/research/snd
英文摘要：
Large-scale vision-language models (VLMs) have shown a strong zero-shot generalization capability on unseen-domain data. However, adapting pre-trained VLMs to a sequence of downstream tasks often leads to the forgetting of previously learned knowledge and a reduction in zero-shot classification performance. To tackle this problem, we propose a unique Selective Dual-Teacher Knowledge Transfer framework that leverages the most recent fine-tuned and the original pre-trained VLMs as dual teachers to preserve the previously learned knowledge and zero-shot capabilities, respectively. With only access to an unlabeled reference dataset, our proposed framework performs a selective knowledge distillation mechanism by measuring the feature discrepancy from the dual-teacher VLMs. Consequently, our selective dual-teacher knowledge distillation mitigates catastrophic forgetting of previously learned knowledge while preserving the zero-shot capabilities of pre-trained VLMs. Extensive experiments on benchmark datasets demonstrate that our framework is favorable against state-of-the-art continual learning approaches for preventing catastrophic forgetting and zero-shot degradation. Project page: https://chuyu.org/research/snd
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
