通过超鞅实现重尾损失的 PAC-Bayes 泛化界
PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales
AI总结：
针对重尾损失下 PAC-Bayes 泛化界不足的问题，本文仅以有界方差为假设，利用超鞅的马尔可夫不等式扩展，统一给出无界鞅、批量及在线学习界。
摘要：
尽管 PAC-Bayes 如今已成为适用于轻尾损失（例如次高斯或次指数损失）的成熟学习框架，但其向重尾损失情形的扩展在很大程度上仍未被探索，并在近年来引起了越来越多的关注。我们在仅假设损失函数具有有界方差的条件下，给出了重尾损失的 PAC-Bayes 泛化界。在该假设下，我们扩展了 Kuzborskij 等人于2019年以 Efron 相关工作给出的先前结果。我们的关键技术贡献是利用马尔可夫不等式针对超鞅的一种扩展。我们的证明技术统一并扩展了不同的 PAC-Bayesian 框架，既提供无界鞅的界，也提供重尾损失下批量学习和在线学习的界。
英文摘要：
While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years. We contribute PAC-Bayes generalisation bounds for heavy-tailed losses under the sole assumption of bounded variance of the loss function. Under that assumption, we extend previous results from \citet{kuzborskij2019efron}. Our key technical contribution is exploiting an extention of Markov's inequality for supermartingales. Our proof technique unifies and extends different PAC-Bayesian frameworks by providing bounds for unbounded martingales as well as bounds for batch and online learning with heavy-tailed losses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
