CRoF：基于CLIP的鲁棒少样本学习以应对噪声标签
CRoF: CLIP-based Robust Few-shot Learning on Noisy Labels
AI总结：
针对少样本学习中噪声标签导致CLIP分类鲁棒性差的问题，提出即插即用模块CRoF，通过任务导向提示生成器和加权多标签微调策略，提升不同噪声条件下的分类性能。
摘要：
在新领域中，由于特征不精确，噪声标签威胁到少样本学习（FSL）的鲁棒性。CLIP作为一种大规模视觉-语言模型，在基于图像-文本嵌入相似性的FSL任务中表现良好，但容易受到噪声标签引起的错误分类影响。如何增强CLIP在FSL任务中处理噪声数据时的领域泛化能力是一个关键挑战。本文提出了一种新视角来减轻噪声标签的影响，即基于CLIP的鲁棒少样本学习（CRoF）。CRoF是面向CLIP模型的通用即插即用模块。为避免错误分类和混淆的标签嵌入，我们设计了面向少样本任务的提示生成器，为每个类别提供更具判别性的描述。所提出的提示实现了更大的类间文本嵌入距离。此外，我们没有完全信任CLIP的零样本分类，而是在新领域的噪声少样本数据上采用类似标签平滑的加权策略对CLIP进行微调。多个潜在正确标签的权重考虑了CLIP先验知识与原始标签信息之间的关系，以确保可靠性。我们的多标签损失函数进一步支持该范式下的鲁棒训练。综合实验表明，CRoF作为即插即用模块，在不同噪声类型和噪声比例下均优于微调和原始CLIP模型。
英文摘要：
Noisy labels threaten the robustness of few-shot learning (FSL) due to the inexact features in a new domain. CLIP, a large-scale vision-language model, performs well in FSL on image-text embedding similarities, but it is susceptible to misclassification caused by noisy labels. How to enhance domain generalization of CLIP on noisy data within FSL tasks is a critical challenge. In this paper, we provide a novel view to mitigate the influence of noisy labels, CLIP-based Robust Few-shot learning (CRoF). CRoF is a general plug-in module for CLIP-based models. To avoid misclassification and confused label embedding, we design the few-shot task-oriented prompt generator to give more discriminative descriptions of each category. The proposed prompt achieves larger distances of inter-class textual embedding. Furthermore, rather than fully trusting zero-shot classification by CLIP, we fine-tune CLIP on noisy few-shot data in a new domain with a weighting strategy like label-smooth. The weights for multiple potentially correct labels consider the relationship between CLIP's prior knowledge and original label information to ensure reliability. Our multiple label loss function further supports robust training under this paradigm. Comprehensive experiments show that CRoF, as a plug-in, outperforms fine-tuned and vanilla CLIP models on different noise types and noise ratios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
