arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.11071cs.LGstat.ML
收藏

低温下高概率Gibbs算法的泛化

Generalization of the Gibbs algorithm with high probability at low temperatures

Istituto Italiano di Tecnologia(意大利技术研究院)

Andreas Maurer

AI总结:

本文给出了Gibbs算法泛化误差的界,覆盖高温及依赖数据相关损失景观的低温范围,证明了泛化误差随相似或更小经验误差的假设总先验体积减小而减小,为平坦极小值有益性提供理论支持。

摘要:

本文给出了Gibbs算法泛化误差的界,该界恢复了高温范围内已知的数据无关界,并扩展到泛化严重依赖于数据相关损失景观的低温范围。研究表明,高概率下,从Gibbs后验中抽取的单个假设的泛化误差随所有具有相似或更小经验误差的假设的总先验体积的减小而减小。这为平坦极小值有益的信念提供了理论支持。文中讨论了零温极限,并将该界扩展到一类类似的随机算法。

英文摘要:

The paper gives a bound on the generalization error of the Gibbs algorithm, which recovers known data-independent bounds for the high temperature range and extends to the low-temperature range, where generalization depends critically on the data-dependent loss-landscape. It is shown, that with high probability the generalization error of a single hypothesis drawn from the Gibbs posterior decreases with the total prior volume of all hypotheses with similar or smaller empirical error. This gives theoretical support to the belief in the benefit of flat minima. The zero temperature limit is discussed and the bound is extended to a class of similar stochastic algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑