arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.02658cs.LGstat.ML
收藏

特权经验风险最小化的容量极限

On the Capacity Limits of Privileged ERM

Ben-Gurion University of the Negev(内盖夫本-古里安大学)

Michal Sharoni, Sivan Sabato

AI总结:

本研究分析特权经验风险最小化(Privileged ERM)的容量极限,通过反例驳斥既有VC维论断,给出正确上下界,并证明其最坏情况泛化保证不优于非特权方法,除非特权信息容量较小。

摘要:

我们研究了由Vapnik和Vashist(2009)首次提出的监督学习范式——利用特权信息学习。在该范式中,除了样本和标签外,仅对训练样本提供额外的(特权)信息。目标是利用这些信息提高所得分类器的分类精度,而该分类器只能利用新样本实例的非特权信息来预测其标签。我们研究了在Pechyony和Vapnik(2010a)提出的自然特权经验风险最小化算法下,使用零一损失的特权学习理论。我们对该工作中关于该问题所引发的损失类VC维数的一个论断给出了反例;我们得出结论,该论断是不正确的。随后,我们提供了正确的VC维数分析,给出了特权经验风险最小化损失类容量的下界和上界。我们进一步通过泛化分析表明,除非特权信息的容量与非特权信息的容量相似或更小,否则特权经验风险最小化的最坏情况保证无法优于标准的非特权经验风险最小化。这一结果指出了特权经验风险最小化方法的一个重要局限性。在最后的讨论中,我们提出了即使在特权信息容量较大时,特权经验风险最小化仍可能有所帮助的另一种方式。

英文摘要:

We study the supervised learning paradigm called Learning Using Privileged Information, first suggested by Vapnik and Vashist (2009). In this paradigm, in addition to the examples and labels, additional (privileged) information is provided only for training examples. The goal is to use this information to improve the classification accuracy of the resulting classifier, where this classifier can only use the non-privileged information of new example instances to predict their label. We study the theory of privileged learning with the zero-one loss under the natural Privileged ERM algorithm proposed in Pechyony and Vapnik (2010a). We provide a counter example to a claim made in that work regarding the VC dimension of the loss class induced by this problem; We conclude that the claim is incorrect. We then provide a correct VC dimension analysis which gives both lower and upper bounds on the capacity of the Privileged ERM loss class. We further show, via a generalization analysis, that worst-case guarantees for Privileged ERM cannot improve over standard non-privileged ERM, unless the capacity of the privileged information is similar or smaller to that of the non-privileged information. This result points to an important limitation of the Privileged ERM approach. In our closing discussion, we suggest another way in which Privileged ERM might still be helpful, even when the capacity of the privileged information is large.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑