arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.10201cs.LGstat.ML
收藏

稀疏性和DAG约束在线性DAG学习中的作用

On the Role of Sparsity and DAG Constraints for Learning Linear DAGs

University of Toronto(多伦多大学);Johns Hopkins University(约翰斯·霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学)

Ignavier Ng, AmirEmad Ghassami, Kun Zhang

摘要:

基于有向无环图(DAG)的图结构学习是一个具有挑战性的问题,部分原因是可能的图的大搜索空间。最近的一系列工作制定的结构学习问题作为一个连续的约束优化任务,使用最小二乘目标和代数特征的DAG。然而,制定需要一个硬DAG约束,并可能导致优化困难。在本文中,我们研究了在线性高斯和非高斯的情况下,学习DAG模型的稀疏性和DAG约束的渐近作用,并调查他们在有限样本制度的有用性。基于理论结果,我们制定了一个基于似然的评分函数,并表明,只需要应用软稀疏性和DAG约束来学习DAG相当于地面真理DAG。这导致了一个无约束的优化问题,更容易解决。使用基于梯度的优化和GPU加速,我们的程序可以轻松处理数千个节点,同时保持高精度。大量的实验验证了我们提出的方法的有效性,并表明DAG惩罚的似然目标确实是有利的最小二乘与硬DAG约束。

英文摘要:

Learning graphical structures based on Directed Acyclic Graphs (DAGs) is a challenging problem, partly owing to the large search space of possible graphs. A recent line of work formulates the structure learning problem as a continuous constrained optimization task using the least squares objective and an algebraic characterization of DAGs. However, the formulation requires a hard DAG constraint and may lead to optimization difficulties. In this paper, we study the asymptotic role of the sparsity and DAG constraints for learning DAG models in the linear Gaussian and non-Gaussian cases, and investigate their usefulness in the finite sample regime. Based on the theoretical results, we formulate a likelihood-based score function, and show that one only has to apply soft sparsity and DAG constraints to learn a DAG equivalent to the ground truth DAG. This leads to an unconstrained optimization problem that is much easier to solve. Using gradient-based optimization and GPU acceleration, our procedure can easily handle thousands of nodes while retaining a high accuracy. Extensive experiments validate the effectiveness of our proposed method and show that the DAG-penalized likelihood objective is indeed favorable over the least squares one with the hard DAG constraint.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑