arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.01902math.STcs.LGstat.MLstat.TH
收藏

$α-$ Rény-近似后验的渐近相合性

Asymptotic Consistency of $α-$Rényi-Approximate Posteriors

School of Industrial Engineering, Purdue University(普渡大学工业工程学院);Department of Statistics, Purdue University(普渡大学统计系)

Prateek Jaiswal, Vinayak A. Rao, Harsha Honnappa

摘要:

我们研究了$α$ -Rényi近似后验的渐近一致性,这是一类变分贝叶斯方法,它用一个易于处理的分布族中的一个成员来近似一个难以处理的贝叶斯后验,选择该成员以最小化$α$ -Rényi与真实后验的偏差。独特的是,我们的工作是,我们认为设置与$α> 1$,从而在近似值上限的对数似然,因此有更广泛的传播比传统的变分方法,最大限度地减少Kullback-Liebler(KL)的分歧后。我们的主要结果确定了充分条件下,一致性举行,围绕着存在一个“好”的序列的分布在近似家庭,拥有,除其他外,权利收敛速率的极限分布。我们进一步刻画了好的序列,证明了一个序列的分布,收敛太快,不能是一个好的序列。我们还将我们的分析扩展到$α$等于1的设置,对应于反向KL发散的最小值,以及具有局部潜变量的模型。我们还用一些例子说明了好序列的存在性。我们的研究结果补充了越来越多的工作集中在变分贝叶斯方法的频率属性。

英文摘要:

We study the asymptotic consistency properties of $α$-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the $α$-Rényi divergence from the true posterior. Unique to our work is that we consider settings with $α> 1$, resulting in approximations that upperbound the log-likelihood, and consequently have wider spread than traditional variational approaches that minimize the Kullback-Liebler (KL) divergence from the posterior. Our primary result identifies sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family that possesses, among other properties, the right rate of convergence to a limit distribution. We further characterize the good sequence by demonstrating that a sequence of distributions that converges too quickly cannot be a good sequence. We also extend our analysis to the setting where $α$ equals one, corresponding to the minimizer of the reverse KL divergence, and to models with local latent variables. We also illustrate the existence of good sequence with a number of examples. Our results complement a growing body of work focused on the frequentist properties of variational Bayesian methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑