NLPrompt: 噪声标签提示学习用于视觉-语言模型
NLPrompt: Noise-Label Prompt Learning for Vision-Language Models
AI总结：
NLPrompt通过结合PromptMAE和PromptOT，提升视觉-语言模型在噪声标签下的提示学习鲁棒性与精度。
摘要：
视觉-语言基础模型的出现，如CLIP，已彻底改变了图像-文本表示，通过提示学习实现了广泛的应用。尽管其具有潜力，但现实中的数据集往往包含噪声标签，这会降低提示学习的性能。在本文中，我们证明在提示学习中使用均绝对误差（MAE）损失，称为PromptMAE，可以显著提高对噪声标签的鲁棒性，同时保持高精度。虽然MAE简单且因其鲁棒性而被认可，但由于其收敛缓慢和在提示学习之外场景中的表现不佳，它在噪声标签学习中很少被使用。为了阐明PromptMAE的鲁棒性，我们利用特征学习理论，表明MAE可以抑制噪声样本的影响，从而提高信噪比并增强整体鲁棒性。此外，我们引入PromptOT，一种基于提示的最优传输数据净化方法，以进一步增强鲁棒性。PromptOT利用视觉-语言模型中的文本特征作为原型，构建一个最优传输矩阵。该矩阵能够将数据集分为干净和噪声子集，允许对干净子集应用交叉熵损失，对噪声子集应用MAE损失。我们的噪声标签提示学习方法，称为NLPrompt，提供了一种简单且高效的途径，利用视觉-语言模型的表达性表示和精确对齐能力进行鲁棒的提示学习。我们通过在各种噪声设置下进行广泛的实验来验证NLPrompt，展示了显著的性能提升。
英文摘要：
The emergence of vision-language foundation models, such as CLIP, has revolutionized image-text representation, enabling a broad range of applications via prompt learning. Despite its promise, real-world datasets often contain noisy labels that can degrade prompt learning performance. In this paper, we demonstrate that using mean absolute error (MAE) loss in prompt learning, named PromptMAE, significantly enhances robustness against noisy labels while maintaining high accuracy. Though MAE is straightforward and recognized for its robustness, it is rarely used in noisy-label learning due to its slow convergence and poor performance outside prompt learning scenarios. To elucidate the robustness of PromptMAE, we leverage feature learning theory to show that MAE can suppress the influence of noisy samples, thereby improving the signal-to-noise ratio and enhancing overall robustness. Additionally, we introduce PromptOT, a prompt-based optimal transport data purification method to enhance the robustness further. PromptOT employs text features in vision-language models as prototypes to construct an optimal transportation matrix. This matrix effectively partitions datasets into clean and noisy subsets, allowing for the application of cross-entropy loss to the clean subset and MAE loss to the noisy subset. Our Noise-Label Prompt Learning method, named NLPrompt, offers a simple and efficient approach that leverages the expressive representations and precise alignment capabilities of vision-language models for robust prompt learning. We validate NLPrompt through extensive experiments across various noise settings, demonstrating significant performance improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
