弥合弱监督学习与VLM知识蒸馏：面向高效下游适应的噪声部分标签学习
Bridging Weakly-Supervised Learning and VLM Distillation: Noisy Partial Label Learning for Efficient Downstream Adaptation
AI总结：
本文提出协同一致性正则化框架，通过协同伪标签机制和抗过拟合策略，提升预训练VLMs在噪声部分标签下的下游适应性能。
摘要：
在噪声部分标签学习（NPLL）的背景下，每个训练样本都与多个噪声标注者标注的候选标签相关联。随着高性能预训练视觉-语言模型（VLMs）如CLIP、LLaVA和GPT-4V的出现，利用这些模型替代耗时的手动标注并实现无标注训练已成为一个有前景的研究方向。本文研究从预训练VLMs生成的噪声部分标签进行学习，并提出一种协同一致性正则化（Co-Reg）框架。与传统噪声标签学习中假设的对称噪声不同，VLM生成的噪声是实例依赖性的，并反映了预训练模型的内在偏差，带来了更大的挑战。为了解决这个问题，我们联合训练两个神经网络，通过协同伪标签机制进行协同标签净化，同时在标签和特征表示空间中施加一致性正则化。此外，还引入了多种抗过拟合策略，包括对比表示和伪标签的交替优化，以及在共享特征空间中维护类别原型。所提出的方法还可以进一步结合少量手动标注的标签以提升性能。在各种设置下的广泛实验展示了我们方法的有效性，并突显了将弱监督学习整合到预训练模型知识蒸馏中的潜力。
英文摘要：
In the context of noisy partial label learning (NPLL), each training sample is associated with a set of candidate labels annotated by multiple noisy annotators. With the emergence of high-performance pre-trained vision-language models (VLMs) such as CLIP, LLaVA, and GPT-4V, leveraging these models to replace time-consuming manual annotation and enable annotation-free training has become a promising research direction. This paper studies learning from noisy partial labels generated by pre-trained VLMs and proposes a collaborative consistency regularization (Co-Reg) framework. Unlike symmetric noise commonly assumed in traditional noisy label learning, VLM-generated noise is instance-dependent and reflects the intrinsic biases of pre-trained models, posing greater challenges. To address this issue, we jointly train two neural networks to perform collaborative label purification via a co-pseudo-labeling mechanism, while enforcing consistency regularization in both label and feature representation spaces. In addition, multiple anti-overfitting strategies are introduced, including alternating optimization of contrastive representations and pseudo-labels, as well as maintaining class prototypes in a shared feature space. The proposed method can further incorporate few-shot manually annotated labels for performance enhancement. Extensive experiments under various settings demonstrate the effectiveness of our approach and highlight the potential of integrating weakly supervised learning into the knowledge distillation of pre-trained models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
