arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.13148stat.MEstat.COstat.ML
收藏

高维混合模型的部分分解变分推断

Partially factorized variational inference for high-dimensional mixed models

Max Goplerud, Omiros Papaspiliopoulos, Giacomo Zanella

AI总结:

针对广义线性混合模型高维积分估计的计算挑战与朴素变分推断不确定性量化不可靠问题,本文证明标准平均场方法会显著低估后验不确定性,提出放宽平均场假设的部分分解变分推断方法,实现线性计算缩放与可靠不确定性量化,且在R包中实现。

摘要:

广义线性混合模型是应用统计学中的基础工具,但许多设定(如包含多水平分类因子或交互项的设定)因需要计算或近似高维积分而在估计时面临计算挑战。变分推断是执行此类计算的常用方法,尤其在贝叶斯框架下。然而,此类方法的朴素使用会导致不可靠的不确定性量化。我们证明混合模型确实存在这一问题:标准平均场变分推断在高维情形下会显著低估后验不确定性。随后,我们展示了如何通过适当放宽平均场假设,得到不确定性量化不会随维度升高而恶化、总计算成本与参数及观测数量呈线性缩放的方法。我们的理论和数值结果聚焦于具有高斯或二项似然的混合模型,依赖随机图论的联系以获得精确的高维渐近分析。我们还提供了具有独立研究价值的通用结果,将变分推断的准确性与用于求解它的相应坐标上升算法的收敛速度关联起来。我们提出的方法已在R包中实现,详见https://github.com/mgoplerud/vglmer。模拟和真实数据示例的数值结果表明,与多种替代方法相比,我们的方法在计算成本与准确性之间取得了更优的权衡。

英文摘要:

While generalized linear mixed models are a fundamental tool in applied statistics, many specifications, such as those involving categorical factors with many levels or interaction terms, can be computationally challenging to estimate due to the need to compute or approximate high-dimensional integrals. Variational inference is a popular way to perform such computations, especially in the Bayesian context. However, naive use of such methods can provide unreliable uncertainty quantification. We show that this is indeed the case for mixed models, proving that standard mean-field variational inference dramatically underestimates posterior uncertainty in high-dimensions. We then show how appropriately relaxing the mean-field assumption leads to methods whose uncertainty quantification does not deteriorate in high-dimensions, and whose total computational cost scales linearly with the number of parameters and observations. Our theoretical and numerical results focus on mixed models with Gaussian or binomial likelihoods, and rely on connections to random graph theory to obtain sharp high-dimensional asymptotic analysis. We also provide generic results, which are of independent interest, relating the accuracy of variational inference to the convergence rate of the corresponding coordinate ascent algorithm that is used to find it. Our proposed methodology is implemented in the R package, see https://github.com/mgoplerud/vglmer . Numerical results with simulated and real data examples illustrate the favourable computation cost versus accuracy trade-off of our approach compared to various alternatives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑