arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.08444cs.LGcs.AIcs.CCcs.CGstat.ML
收藏

部分概念类的PAC可学习性理论

A Theory of PAC Learnability of Partial Concept Classes

Princeton University(普林斯顿大学);Toyota Technological Institute at Chicago(芝加哥丰田工业大学);Technion(以色列理工学院);Google Research(谷歌研究院)

Noga Alon, Steve Hanneke, Ron Holzman, Shay Moran

摘要:

我们扩展了PAC学习的理论,它允许对各种各样的学习任务进行建模,其中数据满足简化学习过程的特殊属性。例如,数据到决策边界的距离不为零的任务。基本而简单的想法是考虑部分概念:这些是可以在空间的某些部分上未定义的函数。当学习部分概念时,我们假设源分布仅在定义部分概念的点上得到支持。通过这种方式,人们可以自然地表达对数据的假设,例如位于低维表面或边缘条件上。相比之下,这是不清楚的,这些假设可以表达的传统的PAC理论。事实上,我们表现出易于学习的部分概念类,证明不能被传统的PAC理论。这也解决了Attias,Kontorovich和Mansour 2019提出的问题。我们的PAC学习的部分概念类的特征,并揭示了算法的景观,这是从根本上不同于经典的。例如,在经典的PAC模型中,学习归结为经验风险最小化(ERM)。与此形成鲜明对比的是,我们表明,ERM原则未能解释部分概念类的可学习性。事实上,我们展示了非常容易学习的类,但是任何学习它们的算法都必须使用具有无限VC维的假设空间。我们还发现,在这种情况下,样本压缩猜想失败。因此,这一理论的特点是不能代表或解决的问题,在传统的方式。我们认为这是一个证据,它可能会提供洞察力的性质,在现实的情况下,经典理论无法解释的可学习性。

英文摘要:

We extend the theory of PAC learning in a way which allows to model a rich variety of learning tasks where the data satisfy special properties that ease the learning process. For example, tasks where the distance of the data from the decision boundary is bounded away from zero. The basic and simple idea is to consider partial concepts: these are functions that can be undefined on certain parts of the space. When learning a partial concept, we assume that the source distribution is supported only on points where the partial concept is defined. This way, one can naturally express assumptions on the data such as lying on a lower dimensional surface or margin conditions. In contrast, it is not at all clear that such assumptions can be expressed by the traditional PAC theory. In fact we exhibit easy-to-learn partial concept classes which provably cannot be captured by the traditional PAC theory. This also resolves a question posed by Attias, Kontorovich, and Mansour 2019. We characterize PAC learnability of partial concept classes and reveal an algorithmic landscape which is fundamentally different than the classical one. For example, in the classical PAC model, learning boils down to Empirical Risk Minimization (ERM). In stark contrast, we show that the ERM principle fails in explaining learnability of partial concept classes. In fact, we demonstrate classes that are incredibly easy to learn, but such that any algorithm that learns them must use an hypothesis space with unbounded VC dimension. We also find that the sample compression conjecture fails in this setting. Thus, this theory features problems that cannot be represented nor solved in the traditional way. We view this as evidence that it might provide insights on the nature of learnability in realistic scenarios which the classical theory fails to explain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑