arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.17604math.STstat.TH
收藏

高维情况下拉普拉斯近似精度的紧界

Tight Bounds on the Laplace Approximation Accuracy in High Dimensions

Anya Katsevich

AI总结:

本文将高维拉普拉斯近似的TV误差分解为主导项O(d/√n)与余项O(d²/n),证明d²≪n条件一般不可改进,并提供更紧且易计算的上界。

摘要:

在贝叶斯推断中,一种广泛使用的计算高维后验积分的技术是使用后验的高斯代理,即所谓的拉普拉斯近似。我们研究在维度 $d$ 随样本量 $n$ 增长的机制下,该近似在总变差(TV)距离方面的精度问题。多项先前工作已经表明,条件 $d^3\ll n$ 足以保证近似的精度。但在最近的一项突破中,Kasprzak 等人(2022)推导出了一个按 $d/\sqrt n$ 缩放的上界。在本工作中,我们通过将 TV 误差分解为一个 $O(d/\sqrt n)$ 的主导阶项和一个 $O(d^2/n)$ 的余项,进一步深化了对拉普拉斯近似误差的理解。这一分解具有深远的意义:首先,我们利用它证明了条件 $d^2\ll n$ 一般无法改进,具体做法是证明对于源自具有高斯设计的逻辑回归的后验,有 TV$\gtrsim d/\sqrt n$。其次,该分解提供了更紧且更易于计算的 TV 误差上界。我们的结果还为在 $d^2\ll n$ 机制下证明 Bernstein-von Mises(BvM)定理以及校正拉普拉斯近似以考虑偏度打开了大门;这些内容在两篇后续工作中展开。

英文摘要:

In Bayesian inference, a widespread technique to compute integrals against a high-dimensional posterior is to use a Gaussian proxy to the posterior known as the Laplace approximation. We address the question of accuracy of the approximation in terms of TV distance, in the regime in which dimension $d$ grows with sample size $n$. Multiple prior works have shown the requirement $d^3\ll n$ is sufficient for accuracy of the approximation. But in a recent breakthrough, Kasprzak et al, 2022 derived an upper bound scaling as $d/\sqrt n$. In this work, we further refine our understanding of the Laplace approximation error by decomposing the TV error into an $O(d/\sqrt n)$ leading order term, and an $O(d^2/n)$ remainder. This decomposition has far reaching implications: first, we use it to prove that the requirement $d^2\ll n$ cannot in general be improved by showing TV$\gtrsim d/\sqrt n$ for a posterior stemming from logistic regression with Gaussian design. Second, the decomposition provides tighter and more easily computable upper bounds on the TV error. Our result also opens the door to proving the BvM in the $d^2\ll n$ regime, and correcting the Laplace approximation to account for skew; this is pursued in two follow-up works.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑