平坦后验对贝叶斯模型平均至关重要
Flat Posterior Does Matter For Bayesian Model Averaging
AI总结：
该研究针对后验平坦性对贝叶斯模型平均（BMA）泛化能力的影响被忽视的问题，提出FP-BMA训练目标及对应迁移学习方案，实验证明其能捕获平坦后验并提升泛化性能。
摘要：
贝叶斯神经网络（BNNs）估计模型参数的后验分布，并在预测时利用后验样本实现贝叶斯模型平均（BMA）。然而，尽管损失景观的平坦性在提升神经网络泛化能力中发挥着关键作用，但其对BMA的影响在很大程度上被忽视了。本研究探究了后验平坦度如何影响BMA的泛化能力，并通过实验证明：（1）大多数近似贝叶斯推理方法无法生成平坦后验；（2）未考虑后验平坦性的BMA预测在提升泛化能力方面效果较差。为解决这一问题，我们提出了平坦后验感知贝叶斯模型平均（FP-BMA），这是一种以有原则的贝叶斯方式显式促进平坦后验的新型训练目标。我们还提出了一种平坦后验感知贝叶斯迁移学习方案，可提升下游任务的泛化能力。实验结果表明，FP-BMA成功捕获了平坦后验，提升了泛化性能。
英文摘要：
Bayesian neural networks (BNNs) estimate the posterior distribution of model parameters and utilize posterior samples for Bayesian Model Averaging (BMA) in prediction. However, despite the crucial role of flatness in the loss landscape in improving the generalization of neural networks, its impact on BMA has been largely overlooked. In this work, we explore how posterior flatness influences BMA generalization and empirically demonstrate that (1) most approximate Bayesian inference methods fail to yield a flat posterior and (2) BMA predictions, without considering posterior flatness, are less effective at improving generalization. To address this, we propose Flat Posterior-aware Bayesian Model Averaging (FP-BMA), a novel training objective that explicitly encourages flat posteriors in a principled Bayesian manner. We also introduce a Flat Posterior-aware Bayesian Transfer Learning scheme that enhances generalization in downstream tasks. Empirically, we show that FP-BMA successfully captures flat posteriors, improving generalization performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
