确定性近似EM算法及其在Riemann近似EM和调和EM中的应用
Deterministic Approximate EM Algorithm; Application to the Riemann Approximation EM and the Tempered EM
摘要:
期望最大化(EM)算法被广泛用于优化具有潜在变量的非凸似然函数。许多作者修改了其简单的设计以适应更具体的情况。例如,期望(E)步骤已被蒙特卡罗(MC),马尔可夫链蒙特卡罗或回火近似等取代。相比之下,文献是缺乏的,当谈到确定性的近似。在本文中,我们介绍了一个理论框架,与国家的最先进的收敛保证,任何确定性近似的E步骤。我们从理论上和经验上分析了几个近似,适合这个框架。首先,对于难处理的E-步骤,我们引入了一个确定性版本的MC-EM使用黎曼和。一种简单的方法,不需要任何超参数微调,当低维不保证MC-EM时有用。然后,我们考虑回火近似,从模拟退火文献中借来的,用来逃避局部极值。我们证明,回火EM验证收敛保证比以前考虑的温度分布范围更广。我们经验性地展示了新的非平凡配置文件如何更成功地逃避对抗性初始化。最后,我们结合黎曼和回火近似成一个方法,实现他们的目的。
英文摘要:
The Expectation Maximisation (EM) algorithm is widely used to optimise non-convex likelihood functions with latent variables. Many authors modified its simple design to fit more specific situations. For instance, the Expectation (E) step has been replaced by Monte Carlo (MC), Markov Chain Monte Carlo or tempered approximations, etc. Most of the well-studied approximations belong to the stochastic class. By comparison, the literature is lacking when it comes to deterministic approximations. In this paper, we introduce a theoretical framework, with state-of-the-art convergence guarantees, for any deterministic approximation of the E step. We analyse theoretically and empirically several approximations that fit into this framework. First, for intractable E-steps, we introduce a deterministic version of MC-EM using Riemann sums. A straightforward method, not requiring any hyper-parameter fine-tuning, useful when the low dimensionality does not warrant a MC-EM. Then, we consider the tempered approximation, borrowed from the Simulated Annealing literature and used to escape local extrema. We prove that the tempered EM verifies the convergence guarantees for a wider range of temperature profiles than previously considered. We showcase empirically how new non-trivial profiles can more successfully escape adversarial initialisations. Finally, we combine the Riemann and tempered approximations into a method that accomplishes both their purposes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。