arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.08649stat.MLcs.LG
收藏

PAC-Bayesian 界实用分解的通用框架

A General Framework for the Practical Disintegration of PAC-Bayesian Bounds

Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Ecole Normale Supérieure(巴黎高等师范学院);PSL Research University(巴黎文理研究大学);Université Laval(拉瓦尔大学);Université Jean Monnet Saint-Étienne(圣艾蒂安让·莫内大学);Institut d Optique Graduate School(光学研究所研究生院);Institut Universitaire de France(法兰西大学研究院)

Paul Viallard, Pascal Germain, Amaury Habrard, Emilie Morvant

AI总结:

本文提出一种新的 PAC-Bayesian 分解界框架,可直接对单个假设给出泛化保证,避免传统去随机化步骤,并在神经网络上取得显著优于现有方法的实际效果。

摘要:

PAC-Bayesian 界在研究随机分类器的泛化能力时以紧致性和信息量著称。然而,当应用于神经网络等某些确定性模型族时,它们需要一个宽松且代价高昂的去随机化步骤。作为该步骤的替代方案,我们提出了新的 PAC-Bayesian 泛化界,其独创性在于提供分解界(disintegrated bounds),即它们对单个假设给出保证,而不是通常的平均分析。我们的界易于优化,并可用于设计学习算法。我们在神经网络上展示了这一行为,并证明了相对于最先进框架的显著实际改进。

英文摘要:

PAC-Bayesian bounds are known to be tight and informative when studying the generalization ability of randomized classifiers. However, they require a loose and costly derandomization step when applied to some families of deterministic models such as neural networks. As an alternative to this step, we introduce new PAC-Bayesian generalization bounds that have the originality to provide disintegrated bounds, i.e., they give guarantees over one single hypothesis instead of the usual averaged analysis. Our bounds are easily optimizable and can be used to design learning algorithms. We illustrate this behavior on neural networks, and we show a significant practical improvement over the state-of-the-art framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑