arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.06715math.DSmath.STstat.TH
收藏

坐标上升变分推理的动力学:二维伊辛模型的一个实例研究

Dynamics of coordinate ascent variational inference: A case study in 2D Ising models

Sean Plummer, Debdeep Pati, Anirban Bhattacharya

摘要:

变分算法在过去的二十年中作为贝叶斯推理的可扩展计算环境而获得了突出地位。在这篇文章中,我们探索工具,从动力系统文献研究收敛的坐标上升算法的平均场变分推理。针对两个节点上的伊辛模型,我们充分刻画了顺序坐标上升算法及其并行版本的动力学特性。我们观察到,在政权的目标函数是凸的,这两个算法是稳定的,并表现出收敛到唯一的不动点。我们的分析揭示了有趣的{ \em不一致}这两个版本的算法在该地区时,目标函数是非凸的。事实上,并行版本表现出周期性的振荡行为,这是不存在的顺序版本。从马尔可夫链蒙特卡洛文献的直觉,我们{ \em经验}表明,伊辛模型的参数扩展,俗称爱德华-索卡尔耦合,导致扩大的制度收敛到全局最优。

英文摘要:

Variational algorithms have gained prominence over the past two decades as a scalable computational environment for Bayesian inference. In this article, we explore tools from the dynamical systems literature to study convergence of coordinate ascent algorithms for mean field variational inference. Focusing on the Ising model defined on two nodes, we fully characterize the dynamics of the sequential coordinate ascent algorithm and its parallel version. We observe that in the regime where the objective function is convex, both the algorithms are stable and exhibit convergence to the unique fixed point. Our analyses reveal interesting {\em discordances} between these two versions of the algorithm in the region when the objective function is non-convex. In fact, the parallel version exhibits a periodic oscillatory behavior which is absent in the sequential version. Drawing intuition from the Markov chain Monte Carlo literature, we {\em empirically} show that a parameter expansion of the Ising model, popularly called as the Edward--Sokal coupling, leads to an enlargement of the regime of convergence to the global optima.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑