关于概率神经网络的冷后验：理解冷后验效应及一种具有紧泛化保证的学习冷后验的新方法
On Cold Posteriors of Probabilistic Neural Networks: Understanding the Cold Posterior Effect and A New Way to Learn Cold Posteriors with Tight Generalization Guarantees
AI总结：
本文研究概率神经网络中的冷后验效应，提出一种具有紧泛化保证的学习冷后验的新方法，通过温度调整平衡似然与先验以提升预测性能。
摘要：
贝叶斯推断提供了一个原则性的概率框架，通过贝叶斯定理基于先验知识和观测数据更新信念来量化不确定性。在贝叶斯深度学习中，神经网络权重被视为具有先验分布的随机变量，从而允许对预测不确定性进行概率解释和量化。然而，贝叶斯方法缺乏对未见数据的理论泛化保证。PAC-贝叶斯分析通过提供一个频率学派框架来推导随机预测器的泛化界，从而弥补了这一局限，进而证明了贝叶斯方法在机器学习中的可靠性。温度 $T$，或逆温度 $λ= \ rac{1}{T}$，最初源于物理学中的统计力学，自然出现在统计推断的各个领域，包括贝叶斯推断和PAC-贝叶斯分析。在贝叶斯推断中，当 $T < 1$（“冷”后验）时，似然被上调权重，导致后验分布更加尖锐。相反，当 $T > 1$（“暖”后验）时，似然被下调权重，导致后验分布更加弥散。通过平衡观测数据和先验正则化的影响，温度调整可以解决贝叶斯模型中的欠拟合或过拟合问题，从而带来更好的预测性能。
英文摘要：
Bayesian inference provides a principled probabilistic framework for quantifying uncertainty by updating beliefs based on prior knowledge and observed data through Bayes' theorem. In Bayesian deep learning, neural network weights are treated as random variables with prior distributions, allowing for a probabilistic interpretation and quantification of predictive uncertainty. However, Bayesian methods lack theoretical generalization guarantees for unseen data. PAC-Bayesian analysis addresses this limitation by offering a frequentist framework to derive generalization bounds for randomized predictors, thereby certifying the reliability of Bayesian methods in machine learning. Temperature $T$, or inverse-temperature $λ= \frac{1}{T}$, originally from statistical mechanics in physics, naturally arises in various areas of statistical inference, including Bayesian inference and PAC-Bayesian analysis. In Bayesian inference, when $T < 1$ (``cold'' posteriors), the likelihood is up-weighted, resulting in a sharper posterior distribution. Conversely, when $T > 1$ (``warm'' posteriors), the likelihood is down-weighted, leading to a more diffuse posterior distribution. By balancing the influence of observed data and prior regularization, temperature adjustments can address issues of underfitting or overfitting in Bayesian models, bringing improved predictive performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
