通过无监督知识蒸馏改进学习提示的零样本泛化能力
Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation
AI总结：
本文提出基于无监督知识蒸馏的提示学习方法KDPL，无需标注样本即可集成至现有技术，在十多个基准上有效提升了VLMs的零样本泛化能力并实现知识迁移。
摘要：
视觉语言模型（VLMs）在未见任务上展现出卓越的零样本泛化能力，但在泛化至数据有限的下游任务时，其性能不及监督方法。提示学习正成为一种用于适配VLMs的参数高效方法，但当前最先进的方法仍需要标注样本。在本文中，我们提出了一种基于从更强大模型进行无监督知识蒸馏的提示学习新方法。我们的方法被称为知识蒸馏提示学习（KDPL），它可以集成到现有的提示学习技术中，并在适配过程中消除了对标注样本的需求。我们在十多个标准基准数据集上的实验表明，KDPL在改进学习提示的零样本域泛化、零样本跨数据集泛化和零样本基类到新类泛化问题方面非常有效。KDPL在适配时不需要任何真实标签，此外我们表明，即使在没有任何训练类名知识的情况下，它也能被用于有效地迁移知识。代码已在 https://github.com/miccunifi/KDPL 公开提供。
英文摘要：
Vision-Language Models (VLMs) demonstrate remarkable zero-shot generalization to unseen tasks, but fall short of the performance of supervised methods in generalizing to downstream tasks with limited data. Prompt learning is emerging as a parameter-efficient method for adapting VLMs, but state-of-the-art approaches require annotated samples. In this paper we propose a novel approach to prompt learning based on unsupervised knowledge distillation from more powerful models. Our approach, which we call Knowledge Distillation Prompt Learning (KDPL), can be integrated into existing prompt learning techniques and eliminates the need for labeled examples during adaptation. Our experiments on more than ten standard benchmark datasets demonstrate that KDPL is very effective at improving generalization of learned prompts for zero-shot domain generalization, zero-shot cross-dataset generalization, and zero-shot base-to-novel class generalization problems. KDPL requires no ground-truth labels for adaptation, and moreover we show that even in the absence of any knowledge of training class names it can be used to effectively transfer knowledge. The code is publicly available at https://github.com/miccunifi/KDPL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
