MoPD：面向视觉语言模型的提示混合蒸馏
MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
AI总结：
针对视觉语言模型软提示学习易过拟合已见类别的问题，提出 MoPD 方法，通过门控网络将人工硬提示知识蒸馏至软提示，显著提升了模型在未见类别上的泛化性能。
摘要：
软提示学习方法在将视觉语言模型（VLMs）适配至下游任务方面十分有效。然而，实证表明现有方法倾向于过拟合已见类别，并在未见类别上性能下降。该局限源于训练数据对已见类别的固有偏差。为解决此问题，我们提出一种新颖的软提示学习方法，名为 Mixture-of-Prompts Distillation (MoPD)，它能有效将人工精心设计的硬提示（即教师提示）中的有用知识迁移至可学习的软提示（即学生提示），从而增强软提示在未见类别上的泛化能力。此外，所提 MoPD 方法利用一个门控网络来学习选择用于提示蒸馏的硬提示。大量实验表明，所提 MoPD 方法优于最先进的基线方法，在未见类别上尤为显著。
英文摘要：
Soft prompt learning methods are effective for adapting vision-language models (VLMs) to downstream tasks. Nevertheless, empirical evidence reveals a tendency of existing methods that they overfit seen classes and exhibit degraded performance on unseen classes. This limitation is due to the inherent bias in the training data towards the seen classes. To address this issue, we propose a novel soft prompt learning method, named Mixture-of-Prompts Distillation (MoPD), which can effectively transfer useful knowledge from hard prompts manually hand-crafted (a.k.a. teacher prompts) to the learnable soft prompt (a.k.a. student prompt), thereby enhancing the generalization ability of soft prompts on unseen classes. Moreover, the proposed MoPD method utilizes a gating network that learns to select hard prompts used for prompt distillation. Extensive experiments demonstrate that the proposed MoPD method outperforms state-of-the-art baselines especially on on unseen classes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
