arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

用于灵活且几何最优嵌入和更快收敛的对比坍缩损失

Contrastive-Collapsed Loss for Flexible and Geometrically Optimal Embeddings and Faster Convergence

Blanca Cano-Camarero, Ángela Fernández-Pascual, José R. Dorronsoro

arXiv 2607.12916首次发表:更新:

发表机构

Departamento de Ingeniería Informática, Universidad Autónoma de Madrid(马德里自治大学信息工程系)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

研究提出CoCo损失函数学习归一化且结构良好的表示,鼓励类内坍缩和类间对比,有更优初始化等优势。实验表明其在表格数据集上性能与现有方法竞争,能促进类聚类和更快收敛,是学习判别性表示的有效目标。

AI 中文摘要

在这项工作中,我们引入了CoCo,一种旨在学习归一化且结构良好表示的损失函数。所提出的损失鼓励类内坍缩和类间对比,同时为神经网络保留足够的灵活性,以近似具有类间大角度分离的几何最优嵌入。我们提供了关于CoCo相对于相关目标(如点回归和交叉熵)的理论分析,表明新提出的损失受益于更接近最优配置的初始化、更多信息性梯度以及对类内表示坍缩的更强激励。在来自OpenML - CC18基准的各种表格数据集上的大量实验表明,CoCo与包括核支持向量机、随机森林、点回归和基于交叉熵的神经网络在内的现有方法具有竞争力的性能。此外,理论论证和实证分析都表明该提议促进了更紧密的类聚类和更快的收敛。这些结果凸显了CoCo损失作为学习判别性表示同时保持有竞争力预测性能的有效目标。

英文摘要

In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-class collapse and inter-class contrast while preserving sufficient flexibility for neural networks to approximate geometrically optimal embeddings with large angular separation between classes. We provide a theoretical analysis positioning CoCo with respect to related objectives such as dot regression and cross-entropy, showing that the new proposed loss benefits from closer initialization to the optimal configuration, more informative gradients, and stronger incentives for class-wise representation collapse. Extensive experiments on diverse tabular datasets from the OpenML-CC18 benchmark show that CoCo achieves competitive performance with state-of-the-art methods, including kernel SVM, Random Forest, dot regression, and cross-entropy-based neural networks. In addition, both theoretical arguments and empirical analyses demonstrate that the proposal promotes tighter class clustering and faster convergence. These results highlight CoCo loss as an effective objective for learning discriminative representations while maintaining competitive predictive performance.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑