SuS-X：视觉-语言模型的免训练、仅名称迁移
SuS-X: Training-Free Name-Only Transfer of Vision-Language Models
AI总结：
本文提出SuS-X，仅凭目标类别名称实现CLIP的免训练迁移，结合SuS与TIP-X，在19个基准的零样本及免训练少样本分类上达到最先进水平。
摘要：
对比语言-图像预训练（CLIP）已成为训练大规模视觉-语言模型的一种简单而有效的方法。CLIP在多种下游任务上展现出令人瞩目的零样本分类和检索能力。然而，要充分发挥其潜力，似乎仍有必要进行微调。对整个CLIP模型进行微调可能耗费大量资源且不稳定。此外，近期旨在规避这种微调需求的方法，仍然需要访问来自目标分布的图像。本文采用一种不同的思路，探索免训练的“仅名称迁移”机制：在该机制中，我们关于下游任务拥有的唯一知识就是下游目标类别的名称。我们提出一种新颖方法SuS-X，它由SuS和TIP-X两个关键组成部分构成，既不需要高强度微调，也不需要昂贵的标注数据。SuS-X在19个基准数据集上取得了最先进的零样本分类结果。我们进一步展示了TIP-X在免训练少样本设置中的效用；在该设置下，相比强大的免训练基线方法，我们再次取得最先进结果。代码位于https://github.com/vishaal27/SuS-X。
英文摘要：
Contrastive Language-Image Pre-training (CLIP) has emerged as a simple yet effective way to train large-scale vision-language models. CLIP demonstrates impressive zero-shot classification and retrieval on diverse downstream tasks. However, to leverage its full potential, fine-tuning still appears to be necessary. Fine-tuning the entire CLIP model can be resource-intensive and unstable. Moreover, recent methods that aim to circumvent this need for fine-tuning still require access to images from the target distribution. In this paper, we pursue a different approach and explore the regime of training-free "name-only transfer" in which the only knowledge we possess about the downstream task comprises the names of downstream target categories. We propose a novel method, SuS-X, consisting of two key building blocks -- SuS and TIP-X, that requires neither intensive fine-tuning nor costly labelled data. SuS-X achieves state-of-the-art zero-shot classification results on 19 benchmark datasets. We further show the utility of TIP-X in the training-free few-shot setting, where we again achieve state-of-the-art results over strong training-free baselines. Code is available at https://github.com/vishaal27/SuS-X.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
