On the Convergence of Continuous Constrained Optimization for Structure Learning
Carnegie Mellon University（卡内基梅隆大学）；Mila, Université de Montréal（蒙特利尔大学米拉研究所）；DeepMind（深度思维）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Ignavier Ng, Sébastien Lachapelle, Nan Rosemary Ke, Simon Lacoste-Julien, Kun Zhang
Recently, structure learning of directed acyclic graphs (DAGs) has been formulated as a continuous optimization problem by leveraging an algebraic characterization of acyclicity. The constrained problem is solved using the augmented Lagrangian method (ALM) which is often preferred to the quadratic penalty method (QPM) by virtue of its standard convergence result that does not require the penalty coefficient to go to infinity, hence avoiding ill-conditioning. However, the convergence properties of these methods for structure learning, including whether they are guaranteed to return a DAG solution, remain unclear, which might limit their practical applications. In this work, we examine the convergence of ALM and QPM for structure learning in the linear, nonlinear, and confounded cases. We show that the standard convergence result of ALM does not hold in these settings, and demonstrate empirically that its behavior is akin to that of the QPM which is prone to ill-conditioning. We further establish the convergence guarantee of QPM to a DAG solution, under mild conditions. Lastly, we connect our theoretical results with existing approaches to help resolve the convergence issue, and verify our findings in light of an empirical comparison of them.
Convergence Rates of Constrained Expected Improvement
National University of Singapore（新加坡国立大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Global Convergence of Average Reward Constrained MDPs with Neural Critic and General Policy Parameterization
Department of Computer Science University of Maryland College Park（计算机科学系大学马里兰大学学院公园）；Indian Institute of Technology, Kanpur（印度理工学院坎浦尔）；Department of Electrical Engineering Indian Institute of Technology, Kanpur（电气工程系印度理工学院坎浦尔）；Department of Industrial Engineering Purdue University West Lafayette（工业工程系普渡大学威斯康星拉法叶）
Algorithm for Constrained Markov Decision Process with Linear Convergence
Humboldt University of Berlin（柏林洪堡大学）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；HSE University（高等经济大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs
University of Tennessee, Knoxville（田纳西大学）；University of Maryland, College Park（马里兰大学）；Apple Inc.（苹果公司）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Southern California（南加州大学）
