增强视觉-语言少样本适应中的负学习
Enhancing Vision-Language Few-Shot Adaptation with Negative Learning
AI总结：
提出SimNL负学习方法，通过发现互补负特征并重加权样本抑制噪声，增强视觉-语言模型少样本适应，在15个数据集上优于现有方法。
摘要：
大规模预训练的视觉-语言模型（VLMs）已展现出令人印象深刻的零样本性能和可迁移性，使其能够以数据高效的方式适应下游任务。然而，当仅有少量标注样本可用时，使VLMs适应以区分特定下游任务中相似类别间的细微差异仍然具有挑战性。在这项工作中，我们提出了一种简单而有效的负学习方法SimNL，以更高效地利用少样本标注样本中的任务特定知识。与以往专注于识别一组代表性正特征以定义“什么是{类别}”的方法不同，SimNL发现了一组互补的负特征，定义“什么不是{类别}”，提供了补充正特征的额外见解，以增强任务特定的识别能力。此外，我们识别出当前的适应方法特别容易受到少样本样本集中潜在噪声的影响。为缓解此问题，我们引入了一种即插即用的少样本实例重加权技术，以抑制噪声异常值并放大干净样本，从而实现更稳定的适应。我们在15个数据集上的广泛实验结果验证了所提出的SimNL在少样本学习和领域泛化任务上均优于现有最先进方法，同时实现了具有竞争力的计算效率。代码可在https://github.com/zhangce01/SimNL获取。
英文摘要：
Large-scale pre-trained Vision-Language Models (VLMs) have exhibited impressive zero-shot performance and transferability, allowing them to adapt to downstream tasks in a data-efficient manner. However, when only a few labeled samples are available, adapting VLMs to distinguish subtle differences between similar classes in specific downstream tasks remains challenging. In this work, we propose a Simple yet effective Negative Learning approach, SimNL, to more efficiently exploit the task-specific knowledge from few-shot labeled samples. Unlike previous methods that focus on identifying a set of representative positive features defining "what is a {CLASS}", SimNL discovers a complementary set of negative features that define "what is not a {CLASS}", providing additional insights that supplement the positive features to enhance task-specific recognition capability. Further, we identify that current adaptation approaches are particularly vulnerable to potential noise in the few-shot sample set. To mitigate this issue, we introduce a plug-and-play few-shot instance reweighting technique to suppress noisy outliers and amplify clean samples for more stable adaptation. Our extensive experimental results across 15 datasets validate that the proposed SimNL outperforms existing state-of-the-art methods on both few-shot learning and domain generalization tasks while achieving competitive computational efficiency. Code is available at https://github.com/zhangce01/SimNL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
