arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.08442math.STcs.LGstat.MLstat.TH
收藏

平均场变分推断与TAP自由能:线性模型中的几何与统计性质

Mean-field variational inference with the TAP free energy: Geometric and statistical properties in linear models

University of California, Berkeley(加州大学伯克利分校);Yale University(耶鲁大学)

Michael Celentano, Zhou Fan, Licong Lin, Song Mei

AI总结:

本研究针对高维贝叶斯线性模型,提出最小化TAP自由能进行平均场变分推断,证明其局部极小值提供一致且校准的后验估计,并给出AMP算法及线性收敛保证。

摘要:

我们研究当样本量n与维度p相当时,贝叶斯线性模型中的平均场变分推断。在高维情况下,常用的从后验分布最小化Kullback-Leibler散度或最大化证据下界的方法,可能偏离真实后验均值并低估后验不确定性。我们转而研究TAP自由能的最小化,在高维渐近框架下证明其存在一个局部极小值,该极小值提供后验边际的一致估计,并可用于正确校准的后验推断。在几何上,我们证明TAP自由能的景观在该局部极小值的一个广泛邻域内是强凸的,在一定的通用条件下,该邻域可通过近似消息传递(AMP)算法找到。随后,我们展示了一种在该局部邻域内线性收敛到极小值的有效算法。在推测没有有效算法能找到该局部邻域的设定中,我们证明了AMP可达到的TAP自由能局部极小值具有类似的几何性质,并表明基于该极小值的后验推断仍然正确校准。

英文摘要:

We study mean-field variational inference in a Bayesian linear model when the sample size n is comparable to the dimension p. In high dimensions, the common approach of minimizing a Kullback-Leibler divergence from the posterior distribution, or maximizing an evidence lower bound, may deviate from the true posterior mean and underestimate posterior uncertainty. We study instead minimization of the TAP free energy, showing in a high-dimensional asymptotic framework that it has a local minimizer which provides a consistent estimate of the posterior marginals and may be used for correctly calibrated posterior inference. Geometrically, we show that the landscape of the TAP free energy is strongly convex in an extensive neighborhood of this local minimizer, which under certain general conditions can be found by an Approximate Message Passing (AMP) algorithm. We then exhibit an efficient algorithm that linearly converges to the minimizer within this local neighborhood. In settings where it is conjectured that no efficient algorithm can find this local neighborhood, we prove analogous geometric properties for a local minimizer of the TAP free energy reachable by AMP, and show that posterior inference based on this minimizer remains correctly calibrated.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑