The Laplacian-constrained Gaussian Markov Random Field (LGMRF) is a common multivariate statistical model for learning a weighted sparse dependency graph from given data. This graph learning problem can be formulated as a maximum likelihood estimation (MLE) of the precision matrix, subject to Laplacian structural constraints, with a sparsity-inducing penalty term. This paper aims to solve this learning problem accurately and efficiently. First, since the commonly used $\ell_1$-norm penalty is inappropriate in this setting and may lead to a complete graph, we employ the nonconvex minimax concave penalty (MCP), which promotes sparse solutions with lower estimation bias. Second, as opposed to existing first-order methods for this problem, we develop a second-order proximal Newton approach to obtain an efficient solver, utilizing several algorithmic features, such as using Conjugate Gradients, preconditioning, and splitting to active/free sets. Numerical experiments demonstrate the advantages of the proposed method in terms of both computational complexity and graph learning accuracy compared to existing methods.
Proximal Newton-type methods for minimizing composite functions
Institute for Computational and Mathematical Engineering, Stanford University(斯坦福大学计算与数学工程研究所);Department of Management Science and Engineering, Stanford University(斯坦福大学管理科学与工程系)
A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problems
Zhejiang University City College(浙大城市学院);Southwest Jiaotong University(西南交通大学);The Hong Kong Polytechnic University(香港理工大学);National University of Singapore(新加坡国立大学)
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
The University of Edinburgh(爱丁堡大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Moscow Institute of Physics and Technology (MIPT)(莫斯科物理技术学院)
A distributed semismooth Newton based augmented Lagrangian method for distributed optimization
School of Mathematics, Southwest Jiaotong University(西南交通大学数学学院);School of Computer and Computing Science, Hangzhou City University(杭州城市大学计算机与计算科学学院);Institute of Mathematics, Zhejiang Wanli University(浙江万里大学数学学院)
Avoiding Communication in Proximal Methods for Convex Optimization Problems
School of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ(电气与计算机工程系,罗格斯大学,新泽西州皮萨迦way);EECS Department, University of Berkeley, Berkeley, CA(电气工程与计算机科学系,伯克利大学,加利福尼亚州伯克利);MSIS Department, Rutgers University, Piscataway, NJ(管理科学与信息系,罗格斯大学,新泽西州皮萨迦way)