arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.14796cs.LG
收藏

基于粒子的在线贝叶斯抽样

Particle-based Online Bayesian Sampling

University of California, Santa Barbara(加州大学圣巴巴拉分校);Microsoft Research Asia(微软亚洲研究院)

Yifan Yang, Chang Liu, Zheng Zhang

摘要:

在线优化由于其跟踪真实世界流数据的能力而获得了越来越多的关注。虽然在线优化方法已被广泛研究的设置频率统计,很少有作品考虑在线优化与贝叶斯抽样问题。在本文中,我们研究了在线粒子变分推理(OPVI)算法,使用一组粒子来表示近似分布。为了减少使用随机近似所造成的梯度误差,我们包括一个次线性增加批量的方法,以减少方差。为了跟踪一系列动态变化的目标后验序列的OPVI算法的性能,我们提供了一个详细的理论分析,从Wasserstein梯度流的角度与动态遗憾。综合实验和贝叶斯神经网络实验表明,该算法比单纯应用现有的贝叶斯抽样方法在在线环境下取得了更好的效果。

英文摘要:

Online optimization has gained increasing interest due to its capability of tracking real-world streaming data. Although online optimization methods have been widely studied in the setting of frequentist statistics, few works have considered online optimization with the Bayesian sampling problem. In this paper, we study an Online Particle-based Variational Inference (OPVI) algorithm that uses a set of particles to represent the approximating distribution. To reduce the gradient error caused by the use of stochastic approximation, we include a sublinear increasing batch-size method to reduce the variance. To track the performance of the OPVI algorithm with respect to a sequence of dynamically changing target posterior, we provide a detailed theoretical analysis from the perspective of Wasserstein gradient flow with a dynamic regret. Synthetic and Bayesian Neural Network experiments show that the proposed algorithm achieves better results than naively applying existing Bayesian sampling methods in the online setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑