arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.11449math.STstat.TH
收藏

通过扩散过程在高维中进行后验采样

Posterior Sampling in High Dimension via Diffusion Processes

Andrea Montanari, Yuchen Wu

AI总结:

本文研究基于扩散过程和变分方法的高维后验采样算法,通过欧拉离散化及消息传递近似后验期望,在稀疏回归和低秩矩阵估计中实现了恒定信噪比下的精度保证。

摘要:

从后验分布中采样是贝叶斯统计中的一个关键技术问题。对于常用的马尔可夫链蒙特卡洛算法,很难获得严格的保证。在本文中,我们研究了一类基于扩散过程和变分方法的替代算法。该扩散过程的构造方式使得在其最终时刻,它能近似目标后验分布。该扩散的漂移由在给定数据和额外噪声观测的情况下,未知参数向量 ${\oldsymbol θ}$ 的后验期望给出。为了构建高效的采样算法,我们使用了扩散过程的简单欧拉离散化,并利用消息传递算法和变分推断技术来近似后验期望预言机。我们将此方法应用于高维统计中两个经典问题的后验采样:稀疏回归和尖峰模型内的低秩矩阵估计。在这两种情况下,我们开发了在恒定信噪比机制下具有精度保证的首批算法。

英文摘要:

Sampling from the posterior is a key technical problem in Bayesian statistics. Rigorous guarantees are difficult to obtain for Markov Chain Monte Carlo algorithms of common use. In this paper, we study an alternative class of algorithms based on diffusion processes and variational methods. The diffusion is constructed in such a way that, at its final time, it approximates the target posterior distribution. The drift of this diffusion is given by the posterior expectation of the unknown parameter vector ${\boldsymbol θ}$ given the data and the additional noisy observations. In order to construct an efficient sampling algorithm, we use a simple Euler discretization of the diffusion process, and leverage message passing algorithms and variational inference techniques to approximate the posterior expectation oracle. We apply this method to posterior sampling in two canonical problems in high-dimensional statistics: sparse regression and low-rank matrix estimation within the spiked model. In both cases we develop the first algorithms with accuracy guarantees in the regime of constant signal-to-noise ratios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑