DPA：视觉-语言模型无监督适应的双原型对齐
DPA: Dual Prototypes Alignment for Unsupervised Adaptation of Vision-Language Models
AI总结：
针对视觉-语言模型（如CLIP）在无监督领域适应中因伪标签噪声导致的性能受限问题，提出DPA方法，通过双原型对齐解决视觉文本表示错位，构建准确伪标签，在13个任务上显著优于基线。
摘要：
视觉-语言模型（VLMs），如CLIP，在零样本图像分类中展现出显著潜力。然而，将这些模型适配到新领域仍具挑战，尤其是在无标签数据的无监督场景中。近期研究提出伪标签方法，利用无标签目标数据以无监督方式适配CLIP。但由于CLIP视觉与文本表示错位导致伪标签噪声，这些方法效果受限。本研究提出DPA——一种VLMs无监督领域适应方法，引入双原型概念（作为不同分类器）及输出凸组合构建准确伪标签，通过伪标签排序增强早期自训练鲁棒性，还通过对齐文本与图像原型解决视觉-文本错位问题。13个下游视觉任务实验表明，DPA显著优于零样本CLIP及现有最优无监督适应基线。
英文摘要：
Vision-language models (VLMs), e.g., CLIP, have shown remarkable potential in zero-shot image classification. However, adapting these models to new domains remains challenging, especially in unsupervised settings where labeled data is unavailable. Recent research has proposed pseudo-labeling approaches to adapt CLIP in an unsupervised manner using unlabeled target data. Nonetheless, these methods struggle due to noisy pseudo-labels resulting from the misalignment between CLIP's visual and textual representations. This study introduces DPA, an unsupervised domain adaptation method for VLMs. DPA introduces the concept of dual prototypes, acting as distinct classifiers, along with the convex combination of their outputs, thereby leading to accurate pseudo-label construction. Next, it ranks pseudo-labels to facilitate robust self-training, particularly during early training. Finally, it addresses visual-textual misalignment by aligning textual prototypes with image prototypes to further improve the adaptation performance. Experiments on 13 downstream vision tasks demonstrate that DPA significantly outperforms zero-shot CLIP and the state-of-the-art unsupervised adaptation baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
