Sampling from the posterior is a key technical problem in Bayesian statistics. Rigorous guarantees are difficult to obtain for Markov Chain Monte Carlo algorithms of common use. In this paper, we study an alternative class of algorithms based on diffusion processes and variational methods. The diffusion is constructed in such a way that, at its final time, it approximates the target posterior distribution. The drift of this diffusion is given by the posterior expectation of the unknown parameter vector ${\boldsymbol θ}$ given the data and the additional noisy observations.
In order to construct an efficient sampling algorithm, we use a simple Euler discretization of the diffusion process, and leverage message passing algorithms and variational inference techniques to approximate the posterior expectation oracle.
We apply this method to posterior sampling in two canonical problems in high-dimensional statistics: sparse regression and low-rank matrix estimation within the spiked model. In both cases we develop the first algorithms with accuracy guarantees in the regime of constant signal-to-noise ratios.
Inference for Multivariate Regression Model based on synthetic data generated under Fixed-Posterior Predictive Sampling: comparison with Plug-in Sampling
Provable Diffusion Posterior Sampling for Bayesian Inversion
Southwestern University of Finance and Economics(西南财经大学);RWTH Aachen University(亚琛工业大学);Wuhan University(武汉大学);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院)