面向对比表示学习的基于正则化最优传输的难负例采样
Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning
AI总结：
该研究针对无监督对比表示学习的难负例采样问题，提出基于正则化最优传输的极小极大框架，通过正则化传输耦合控制负例难度，设计新的基代价函数，提升了下游任务性能。
摘要：
我们研究了无监督对比表示学习中难负例采样分布的设计问题。我们提出并分析了一种新颖的极小极大框架，该框架寻求一种表示，使得在所有耦合（满足边缘约束的正负样本联合分布）上的最大（最坏情况）广义对比学习损失最小，并证明由此得到的极小极大最优表示将是退化的。这为在耦合中引入额外的正则化约束提供了首个理论依据。我们通过最优传输（OT）理论重新阐释了该极小极大问题，并利用正则化传输耦合来控制负例的难度。通过实验我们证明，与基线负分布生成的负样本相比，我们设计的负分布生成的负样本与锚点更相似。我们还证明，熵正则化生成的负采样分布的参数形式与近期一种最先进的负采样设计的参数形式相似，并且在多个数据集上具有相近的性能。利用所揭示的与最优传输的关联，我们提出了一种用于设计负分布的新的基代价函数，并表明与使用平方欧氏代价学习到的表示相比，所学习到的表示在下游任务上的性能有所提升。
英文摘要：
We study the problem of designing hard negative sampling distributions for unsupervised contrastive representation learning. We propose and analyze a novel min-max framework that seeks a representation which minimizes the maximum (worst-case) generalized contrastive learning loss over all couplings (joint distributions between positive and negative samples subject to marginal constraints) and prove that the resulting min-max optimum representation will be degenerate. This provides the first theoretical justification for incorporating additional regularization constraints on the couplings. We re-interpret the min-max problem through the lens of Optimal Transport (OT) theory and utilize regularized transport couplings to control the degree of hardness of negative examples. Through experiments we demonstrate that the negative samples generated from our designed negative distribution are more similar to the anchor than those generated from the baseline negative distribution. We also demonstrate that entropic regularization yields negative sampling distributions with parametric form similar to that in a recent state-of-the-art negative sampling design and has similar performance in multiple datasets. Utilizing the uncovered connection with OT, we propose a new ground cost for designing the negative distribution and show improved performance of the learned representation on downstream tasks compared to the representation learned when using squared Euclidean cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
