arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.21765cs.LG
收藏

Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning

Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning

Department of Statistics, LMU Munich(统计系,慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心)

Emanuel Sommer, David Rügamer

AI总结:

本文探讨了在贝叶斯深度学习中采样推理(SAI)的潜力,指出其在计算效率上已与优化方法相当,并可能成为更有效的推理方法。核心贡献是推动SAI在贝叶斯神经网络中的应用,解决现有误解,以实现更精确的不确定性量化。

摘要:

贝叶斯神经网络(BNNs)中基于采样的推理(SAI)的实用应用仍然有限,部分原因是持续存在的关于其可行性和效率的误解。本文认为,SAI在计算上已与基于优化的方法达到平衡,并即将超越这些方法,成为BNNs中更有效和高效的推理方法。这一发展应成为整个社区的利益,推动BNNs作为一种原则性的范式,实现其长期未实现的承诺,即为神经网络提供原则性的不确定性量化。SAI甚至可以做到更多——通过模型平均获得更优的预测性能,成为各种可能的下游任务的基础,并为BNNs的景观提供关键见解。为了实现这种变革并释放采样的潜力,克服当前的误解是必要的第一步。下一步是重新定向研究努力,解决SAI中尚存的挑战。特别是,社区必须专注于两个核心问题:充分探索后验景观和高保真度地蒸馏后验样本以实现高效的下游推理。通过解决概念和实践上的障碍,我们可以解锁SAI的全部潜力,并将其确立为贝叶斯深度学习中的核心工具。

英文摘要:

The practical adoption of sampling-based inference (SAI) in Bayesian neural networks (BNNs) remains limited, partly due to persistent misconceptions about the feasibility and efficiency of sampling. This position paper argues that SAI has achieved computational parity with optimization-based methods and is at the verge of superseding such methods for effective and efficient inference in BNNs. This development should be in the interest of the whole community, promoting BNNs as a principled paradigm with its long-standing yet unfulfilled promise of providing principled uncertainty quantification for neural networks. SAI can even do more -- yielding superior prediction performance through model averaging, serving as the foundation for a plethora of possible downstream tasks, and providing crucial insights into the landscape of BNNs. In order to make such a change happen and unfold the potential of sampling, overcoming current misconceptions is a necessary first step. The next step is to realign research efforts toward addressing remaining challenges in SAI. In particular, the community must focus on two core problems: sufficient exploration of the posterior landscape and high-fidelity distillation of posterior samples for efficient downstream inference. By addressing conceptual and practical obstacles, we can unlock the full potential of SAI and establish it as a central tool in Bayesian deep learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑