在近端环境中通过阶段训练进行成对学习
Pairwise Learning via Stagewise Training in Proximal Setting
摘要：
成对目标范式是机器学习的一个重要和基本方面。使用成对目标函数的机器学习方法的示例包括人脸识别中的差分网络、度量学习、二分学习、多核学习和曲线下面积（AUC）最大化。与逐点学习相比，成对学习的样本大小与样本数量成二次方增长，因此其复杂度也随之增加。研究人员大多通过利用在线学习系统来解决这一挑战。然而，最近的研究提供了自适应样本大小训练光滑损失函数作为一个更好的策略，在收敛性和复杂性，但没有一个全面的理论研究。在一个独特的研究路线，重要性采样引发了相当大的兴趣，有限逐点和最小化。这是因为随机梯度方差，这导致收敛速度大大减慢。在本文中，我们结合自适应样本大小和重要性采样技术的成对学习，与非光滑凸成对损失函数的收敛保证。特别地，使用扩展的训练集随机地训练模型用于从稳定性界限导出的预定义次数的迭代。此外，我们证明了在每次迭代中对相反的实例进行采样可以降低梯度的方差，从而加速收敛。在AUC最大化的各种数据集上的实验证实了理论结果。
英文摘要：
The pairwise objective paradigms are an important and essential aspect of machine learning. Examples of machine learning approaches that use pairwise objective functions include differential network in face recognition, metric learning, bipartite learning, multiple kernel learning, and maximizing of area under the curve (AUC). Compared to pointwise learning, pairwise learning's sample size grows quadratically with the number of samples and thus its complexity. Researchers mostly address this challenge by utilizing an online learning system. Recent research has, however, offered adaptive sample size training for smooth loss functions as a better strategy in terms of convergence and complexity, but without a comprehensive theoretical study. In a distinct line of research, importance sampling has sparked a considerable amount of interest in finite pointwise-sum minimization. This is because of the stochastic gradient variance, which causes the convergence to be slowed considerably. In this paper, we combine adaptive sample size and importance sampling techniques for pairwise learning, with convergence guarantees for nonsmooth convex pairwise loss functions. In particular, the model is trained stochastically using an expanded training set for a predefined number of iterations derived from the stability bounds. In addition, we demonstrate that sampling opposite instances at each iteration reduces the variance of the gradient, hence accelerating convergence. Experiments on a broad variety of datasets in AUC maximization confirm the theoretical results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
