利用正交原型防止对比学习中的坍缩（CLOP）
Preventing Collapse in Contrastive Learning with Orthonormal Prototypes (CLOP)
AI总结：
针对对比学习中的神经坍缩问题，该研究分析大学习率对余弦对比损失的影响并推导缓解界，提出聚焦子空间分离的半监督损失CLOP，实验证明其可提升性能与稳定性。
摘要：
对比学习已成为深度学习中一种强大的方法，擅长通过对比来自不同分布的样本来学习有效的表征。然而，神经坍缩（即嵌入向量收敛到低维空间的现象）构成了重大挑战，在半监督和自监督设置中尤为突出。本文首先从理论上分析了大学习率对仅依赖余弦相似度指标的对比损失的影响，并推导出一个可缓解这种坍缩的理论界。基于这些见解，我们提出了CLOP——一种新型半监督损失函数，旨在通过促进类别嵌入之间形成正交线性子空间来防止神经坍缩。与此前强制采用单纯形ETF结构的方法不同，CLOP聚焦于子空间分离，可生成更具区分度的嵌入。通过在真实数据集和合成数据集上开展的大量实验，我们证明CLOP提升了性能，在不同学习率和批次大小下均具备更强的稳定性。
英文摘要：
Contrastive learning has emerged as a powerful method in deep learning, excelling at learning effective representations through contrasting samples from different distributions. However, neural collapse, where embeddings converge into a lower-dimensional space, poses a significant challenge, especially in semi-supervised and self-supervised setups. In this paper, we first theoretically analyze the effect of large learning rates on contrastive losses that solely rely on the cosine similarity metric, and derive a theoretical bound to mitigate this collapse. {Building on these insights, we propose CLOP, a novel semi-supervised loss function designed to prevent neural collapse by promoting the formation of orthogonal linear subspaces among class embeddings.} Unlike prior approaches that enforce a simplex ETF structure, CLOP focuses on subspace separation, leading to more distinguishable embeddings. Through extensive experiments on real and synthetic datasets, we demonstrate that CLOP enhances performance, providing greater stability across different learning rates and batch sizes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
