困难示例损害无监督对比学习：一种理论视角
Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective
AI总结：
本文从理论角度探讨了困难示例对无监督对比学习的影响，发现移除困难示例可提升下游分类性能，并通过理论分析和实验证明了其有效性。
摘要：
无监督对比学习近年来在各种任务中展现出显著的性能提升，往往接近甚至超越监督学习。然而，其学习机制本质上与监督学习不同。先前的研究表明，在监督学习中被广泛认可的困难示例（即决策边界附近的示例）在无监督设置中贡献很小。在本文中，或许令人惊讶的是，我们发现直接移除困难示例虽然会减少样本量，但可以提升对比学习的下游分类性能。为了揭示背后的原因，我们开发了一个理论框架，用于建模不同样本对之间的相似性。基于这一框架，我们进行了深入的理论分析，揭示了困难示例的存在会负面影响对比学习的泛化能力。此外，我们证明了移除这些示例以及边距调节和温度缩放等技术可以增强其泛化界限，从而提升性能。实证上，我们提出了一种简单且高效的机制来选择困难示例，并验证了上述方法的有效性，这证实了我们所提出理论框架的可靠性。
英文摘要：
Unsupervised contrastive learning has shown significant performance improvements in recent years, often approaching or even rivaling supervised learning in various tasks. However, its learning mechanism is fundamentally different from supervised learning. Previous works have shown that difficult examples (well-recognized in supervised learning as examples around the decision boundary), which are essential in supervised learning, contribute minimally in unsupervised settings. In this paper, perhaps surprisingly, we find that the direct removal of difficult examples, although reduces the sample size, can boost the downstream classification performance of contrastive learning. To uncover the reasons behind this, we develop a theoretical framework modeling the similarity between different pairs of samples. Guided by this framework, we conduct a thorough theoretical analysis revealing that the presence of difficult examples negatively affects the generalization of contrastive learning. Furthermore, we demonstrate that the removal of these examples, and techniques such as margin tuning and temperature scaling can enhance its generalization bounds, thereby improving performance. Empirically, we propose a simple and efficient mechanism for selecting difficult examples and validate the effectiveness of the aforementioned methods, which substantiates the reliability of our proposed theoretical framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
