Large-scale sparse precision matrix estimation has attracted wide interest from the statistics community. The convex partial correlation selection method (CONCORD) developed by Khare et al. (2015) has recently been credited with some theoretical properties for estimating sparse precision matrices. The CONCORD obtains its solution by a coordinate descent algorithm (CONCORD-CD) based on the convexity of the objective function. However, since a coordinate-wise update in CONCORD-CD is inherently serial, a scale-up is nontrivial. In this paper, we propose a novel parallelization of CONCORD-CD, namely, CONCORD-PCD. CONCORD-PCD partitions the off-diagonal elements into several groups and updates each group simultaneously without harming the computational convergence of CONCORD-CD. We guarantee this by employing the notion of edge coloring in graph theory. Specifically, we establish a nontrivial correspondence between scheduling the updates of the off-diagonal elements in CONCORD-CD and coloring the edges of a complete graph. It turns out that CONCORD-PCD simultaneously updates off-diagonal elements in which the associated edges are colorable with the same color. As a result, the number of steps required for updating off-diagonal elements reduces from p(p-1)/2 to p-1 (for even p) or p (for odd p), where p denotes the number of variables. We prove that the number of such steps is irreducible In addition, CONCORD-PCD is tailored to single-instruction multiple-data (SIMD) parallelism. A numerical study shows that the SIMD-parallelized PCD algorithm implemented in graphics processing units (GPUs) boosts the CONCORD-CD algorithm multiple times.
Model identification and local linear convergence of coordinate descent
Université de Bourgogne(勃艮第大学);Institut de Mathématiques de Bourgogne(勃艮第数学研究所);Université Paris-Saclay(巴黎萨克雷大学);Inria(法国国家信息与自动化研究所);CEA(法国原子能委员会);IMAG(格勒诺布尔阿尔卑斯大学综合理工学院(注:IMAG通常指Institut de Mathématiques et d'Informatique de Grenoble,中文常译为“格勒诺布尔数学与信息研究所”,但根据上下文与“Université de Montpellier”关联,此处更准确的常见译法为“格勒诺布尔阿尔卑斯大学数学与信息研究所”,但为简洁,按机构名直译或保留缩写的话,通常Institut de Mathématiques de Grenoble,此处IMAG为其缩写,可译为“格勒诺布尔数学与信息研究所”));Université de Montpellier(蒙彼利埃大学);CNRS(法国国家科学研究中心);IMB(蒙彼利埃大学数学研究所(注:IMB通常指Institut de Mathématiques de Bordeaux,但此处根据上下文与“Université de Bourgogne”关联,应为Institut de Mathématiques de Bourgogne的缩写,即“勃艮第数学研究所”,但原文本中Samuel Vaiter的Affiliation同时有“CNRS and IMB, Université de Bourgogne”,需确认IMB是否为“IMB”,原文本中Samuel Vaiter的Affiliation为“CNRS and IMB, Université de Bourgogne, Dijon, France”,而前面Quentin Klopfenstein的Affiliation是“Université de Bourgogne, Institut de Mathématiques de Bourgogne, Dijon, France”,因此IMB应为“IMB”的独立机构,可能是Institut de Mathématiques de Bourgogne的缩写,即“勃艮第数学研究所”,此处按原文本保留“IMB”,中文译为“勃艮第数学研究所”))