arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.02805math.OCstat.CO
收藏

通过MM算法求解非凸Optimization:收敛性理论

Nonconvex Optimization via MM Algorithms: Convergence Theory

Kenneth Lange, Joong-Ho Won, Alfonso Landeros, Hua Zhou

摘要:

优化最小化(MM)原则是一个非常普遍的框架,用于推导优化算法。它包括期望最大化(EM)算法,邻近梯度算法,凹凸过程,二次下界算法,和邻近距离算法作为特殊情况。除了在统计、优化和成像中的众多应用之外,MM原理在大规模机器学习问题中也有广泛的应用,例如矩阵完备化、判别分析和非负矩阵分解。MM算法在求解非凸最优化问题时具有目标函数凸化、变量分离、数值稳定性和易于实现等优点。然而,与其他优化算法的大量文献相比,MM算法的收敛性分析是分散的和特定的问题。本文给出了MM算法收敛性的统一处理方法。考虑到现代应用,结果包括非光滑目标函数和非渐近分析。

英文摘要:

The majorization-minimization (MM) principle is an extremely general framework for deriving optimization algorithms. It includes the expectation-maximization (EM) algorithm, proximal gradient algorithm, concave-convex procedure, quadratic lower bound algorithm, and proximal distance algorithm as special cases. Besides numerous applications in statistics, optimization, and imaging, the MM principle finds wide applications in large scale machine learning problems such as matrix completion, discriminant analysis, and nonnegative matrix factorizations. When applied to nonconvex optimization problems, MM algorithms enjoy the advantages of convexifying the objective function, separating variables, numerical stability, and ease of implementation. However, compared to the large body of literature on other optimization algorithms, the convergence analysis of MM algorithms is scattered and problem specific. This survey presents a unified treatment of the convergence of MM algorithms. With modern applications in mind, the results encompass non-smooth objective functions and non-asymptotic analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑