arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.07262math.STstat.TH
收藏

高维情形下Laplace近似的精度:一种精细分析与新的偏度校正

The Laplace approximation accuracy in high dimensions: a refined analysis and new skew adjustment

Anya Katsevich

AI总结:

本文通过推导Laplace近似误差的主阶分解,首次提出可证明在高维情形下将精度提升一个数量级的偏度校正,并给出更紧上界与首个下界,证明$d^2\ll n$一般为精度必要条件,最后在Dirichlet后验与高斯设计logistic回归中验证了理论。

摘要:

在贝叶斯推断中,要对感兴趣的参数进行推断,需要从后验分布中采样或计算关于后验分布的积分。在高维情形下,一种常用的降低计算成本的方法是将后验分布替换为其Laplace近似(LA),即一个高斯分布。在本文中,我们推导出Laplace近似误差的主阶分解,这是一种比以往更精确地分析近似精度的有力技术。该技术使我们能够首次推导出对Laplace近似的偏度校正,并证明在高维情形下该校正可将精度提高一个数量级。我们的方法还使我们能够证明标准Laplace近似的更紧上界,以及高维情形下的首个下界。特别地,我们证明了一般而言$d^2\ll n$是Laplace近似具有精度所必需的条件,其中$d$为维度,$n$为样本量。最后,我们将理论应用于两个示例模型:由多项观测产生的Dirichlet后验,以及具有高斯设计的logistic回归。在后一种情形下,我们仅以$d/\sqrt n$的幂次证明了Laplace近似和偏度校正后Laplace近似精度的高概率界。

英文摘要:

In Bayesian inference, making deductions about a parameter of interest requires one to sample from or compute an integral against a posterior distribution. A popular method to make these computations cheaper in high-dimensional settings is to replace the posterior with its Laplace approximation (LA), a Gaussian distribution. In this work, we derive a leading order decomposition of the LA error, a powerful technique to analyze the accuracy of the approximation more precisely than was possible before. It allows us to derive the first ever skew correction to the LA which provably improves its accuracy by an order of magnitude in the high-dimensional regime. Our approach also enables us to prove both tighter upper bounds on the standard LA and the first ever lower bounds in high dimensions. In particular, we prove that $d^2\ll n$ is in general necessary for accuracy of the LA, where $d$ is dimension and $n$ is sample size. Finally, we apply our theory in two example models: a Dirichlet posterior arising from a multinomial observation, and logistic regression with Gaussian design. In the latter setting, we prove high probability bounds on the accuracy of the LA and skew-corrected LA in powers of $d/\sqrt n$ alone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑