探索基于CLIP的分类中的弱到强泛化
Exploring Weak-to-Strong Generalization for CLIP-based Classification
AI总结：
本文提出类别原型学习方法，通过弱监督提升CLIP模型分类性能，实验显示在预训练受限情况下取得显著改进。
摘要：
对齐大规模商业模型与用户意图对于防止有害输出至关重要。当前方法依赖于人工监督，但随着模型复杂性的增加，这种方法变得不切实际。当模型超越人类知识时，提供准确的反馈变得具有挑战性且低效。最近提出的一种新解决方案是使用较弱的模型来监督更强的模型。这一概念利用较弱模型执行评估的能力，从而减少对人工监督者的工作量。先前的工作已经证明了在语言模型背景下弱到强泛化的有效性。将这一概念扩展到视觉-语言模型中，利用这些见解，将已证明的益处适应到多模态环境。在我们的研究中，我们探索了基于CLIP的分类中的弱到强泛化。我们提出了一种方法，即类别原型学习（CPL），旨在通过为每个类别学习更具代表性的原型来增强CLIP模型的分类能力。我们的发现表明，尽管在弱监督下使用简单的损失函数，CPL在目标场景中仍能实现稳健的改进，特别是在预训练受限的情况下。广泛的实验表明，我们的方法在这些设置下是有效的，实现了比强基线方法高出3.67%的改进。
英文摘要：
Aligning large-scale commercial models with user intent is crucial to preventing harmful outputs. Current methods rely on human supervision but become impractical as model complexity increases. When models surpass human knowledge, providing accurate feedback becomes challenging and inefficient. A novel solution proposed recently is using a weaker model to supervise a stronger model. This concept leverages the ability of weaker models to perform evaluations, thereby reducing the workload on human supervisors. Previous work has shown the effectiveness of weak-to-strong generalization in the context of language-only models. Extending this concept to vision-language models leverages these insights, adapting the proven benefits to a multi-modal context. In our study, we explore weak-to-strong generalization for CLIP-based classification. We propose a method, class prototype learning (CPL), which aims to enhance the classification capabilities of the CLIP model, by learning more representative prototypes for each category. Our findings indicate that, despite using a simple loss function under weak supervision, CPL yields robust improvements in targeted scenarios, particularly when pretraining is limited. Extensive experiments demonstrate that our approach is effective under these settings, achieving a 3.67% improvement over strong baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
