重新思考弱监督在助力对比学习中的作用
Rethinking Weak Supervision in Helping Contrastive Learning
AI总结：
针对弱监督信息能否直接用于对比学习的问题，本文建立统一理论框架，通过谱聚类相似度图与误差界分析，揭示半监督标签可改善下游误差而带噪标签作用有限。
摘要：
对比学习在监督学习和无监督学习中均表现出卓越性能，最近还被引入用于解决半监督学习、带噪标签学习等弱监督学习问题。尽管经验证据表明半监督标签能够改善对比学习的表征，但带噪监督信息能否直接用于训练、而非在人工去噪之后使用，仍然未知。因此，为探究半监督信息与带噪标签信息在助力对比学习时的机制差异，我们建立了弱监督下对比学习的统一理论框架。具体而言，我们研究了联合训练监督对比损失与无监督对比损失这一最直观的范式。在谱聚类框架下，我们基于弱标签的后验概率将弱监督信息转化为相似度图，并据此建立了下游分类误差界。我们证明，在该范式下，半监督标签能够改善下游误差界，而带噪标签的作用有限。本文的理论发现为社区重新思考弱监督在助力对比学习中的作用提供了新的见解。
英文摘要：
Contrastive learning has shown outstanding performances in both supervised and unsupervised learning, and has recently been introduced to solve weakly supervised learning problems such as semi-supervised learning and noisy label learning. Despite the empirical evidence showing that semi-supervised labels improve the representations of contrastive learning, it remains unknown if noisy supervised information can be directly used in training instead of after manual denoising. Therefore, to explore the mechanical differences between semi-supervised and noisy-labeled information in helping contrastive learning, we establish a unified theoretical framework of contrastive learning under weak supervision. Specifically, we investigate the most intuitive paradigm of jointly training supervised and unsupervised contrastive losses. By translating the weakly supervised information into a similarity graph under the framework of spectral clustering based on the posterior probability of weak labels, we establish the downstream classification error bound. We prove that semi-supervised labels improve the downstream error bound whereas noisy labels have limited effects under such a paradigm. Our theoretical findings here provide new insights for the community to rethink the role of weak supervision in helping contrastive learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
