分类问题中损失函数与不确定集的一个共轭性质
A Conjugate Property between Loss Functions and Uncertainty Sets in Classification Problems
摘要：
在二元分类问题中，主要有两种方法：损失函数方法和不确定集方法。损失函数方法被应用于主要的学习算法，如支持向量机（SVM）和boosting方法。损失函数表示决策函数对训练样本的惩罚。在学习算法中，损失函数的经验均值最小化以获得分类器。在数学规划发展的背景下，基于损失函数的学习算法被广泛应用于现实世界的数据分析。此外，基于大量的理论工作，这种学习算法的统计特性得到了很好的理解。另一方面，使用所谓的不确定性集的学习方法被用于硬间隔SVM，最小最大概率机（MPM）和最大间隔MPM。在学习算法中，首先根据训练样本为每个二进制标签定义不确定集。然后，两个不确定集之间的最佳分离超平面被用作决策函数。这被认为是最大差值办法的延伸。不确定集方法是鲁棒优化在数学规划领域中的一个应用。不确定集学习算法的统计特性尚未得到深入研究。在本文中，我们考虑这两种方法之间的关系。我们指出，不确定性集是用损失函数的共轭水平集来描述的。基于这种关系，我们研究了使用不确定集的学习算法的统计特性。
英文摘要：
In binary classification problems, mainly two approaches have been proposed; one is loss function approach and the other is uncertainty set approach. The loss function approach is applied to major learning algorithms such as support vector machine (SVM) and boosting methods. The loss function represents the penalty of the decision function on the training samples. In the learning algorithm, the empirical mean of the loss function is minimized to obtain the classifier. Against a backdrop of the development of mathematical programming, nowadays learning algorithms based on loss functions are widely applied to real-world data analysis. In addition, statistical properties of such learning algorithms are well-understood based on a lots of theoretical works. On the other hand, the learning method using the so-called uncertainty set is used in hard-margin SVM, mini-max probability machine (MPM) and maximum margin MPM. In the learning algorithm, firstly, the uncertainty set is defined for each binary label based on the training samples. Then, the best separating hyperplane between the two uncertainty sets is employed as the decision function. This is regarded as an extension of the maximum-margin approach. The uncertainty set approach has been studied as an application of robust optimization in the field of mathematical programming. The statistical properties of learning algorithms with uncertainty sets have not been intensively studied. In this paper, we consider the relation between the above two approaches. We point out that the uncertainty set is described by using the level set of the conjugate of the loss function. Based on such relation, we study statistical properties of learning algorithms using uncertainty sets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
