Multi-Level GNN Preconditioner for Solving Large Scale Problems
Université Paris-Saclay（巴黎萨克雷大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Direction Technologie et Numerique IFP Énergies-Nouvelles（IFP新能源公司技术与数字部）；Safran Tech（赛峰科技）
Large-scale numerical simulations often come at the expense of daunting computations. High-Performance Computing has enhanced the process, but adapting legacy codes to leverage parallel GPU computations remains challenging. Meanwhile, Machine Learning models can harness GPU computations effectively but often struggle with generalization and accuracy. Graph Neural Networks (GNNs), in particular, are great for learning from unstructured data like meshes but are often limited to small-scale problems. Moreover, the capabilities of the trained model usually restrict the accuracy of the data-driven solution. To benefit from both worlds, this paper introduces a novel preconditioner integrating a GNN model within a multi-level Domain Decomposition framework. The proposed GNN-based preconditioner is used to enhance the efficiency of a Krylov method, resulting in a hybrid solver that can converge with any desired level of accuracy. The efficiency of the Krylov method greatly benefits from the GNN preconditioner, which is adaptable to meshes of any size and shape, is executed on GPUs, and features a multi-level approach to enforce the scalability of the entire process. Several experiments are conducted to validate the numerical behavior of the hybrid solver, and an in-depth analysis of its performance is proposed to assess its competitiveness against a C++ legacy solver.
Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Sberbank of Russia（俄罗斯联邦储蓄银行）；Artificial Intelligence Research Institute (AIRI)（人工智能研究所（AIRI））；Fraunhofer Institute for Industrial Mathematics ITWM（弗劳恩霍夫工业数学研究所 ITWM）；Institute of Computational Mathematics and Mathematical Geophysics SB RAS（俄罗斯科学院西伯利亚分院计算数学与数学地球物理研究所）
BandPC: Learning Residual-Band Preconditioner Combinations for Flexible Conjugate Gradient Solvers
Wuhan University of Technology（武汉理工大学）；Embodied Physical Intelligence Center in Civil Engineering, Wuhan University of Technology（武汉理工大学土木工程具身智能中心）；Key Laboratory of Low-Altitude Technology and Smart Urban Renewal, Department of Housing and Urban-Rural Development of Hubei Province, Wuhan University of Technology（湖北省住房和城乡建设厅低空技术与智慧城市更新重点实验室（武汉理工大学））；Sanya Science and Education Innovation Park of Wuhan University of Technology（武汉理工大学三亚科教创新园）
NeuraLSP: A Neural Spectral Preconditioner for Accelerating PDE Solvers
Department of Applied Mathematics and Statistics, Stony Brook University（应用数学与统计学系，史泰尼斯布鲁克大学）；Department of Computer Science, Stony Brook University（计算机科学系，史泰尼斯布鲁克大学）
Expressive Power of Graph Neural Networks for (Mixed-Integer) Quadratic Programs
Department of Mathematics, Massachusetts Institute of Technology（麻省理工学院数学系）；Decision Intelligence Lab, Damo Academy, Alibaba US（达摩院决策智能实验室）；Department of Statistics and Data Science, University of Central Florida（中央佛罗里达大学统计与数据科学系）
Shanghai Jiao Tong University（上海交通大学）；UC San Diego（加州大学圣地亚哥分校）
Current graph neural networks (GNNs) lack generalizability with respect to scales (graph sizes, graph diameters, edge weights, etc..) when solving many graph analysis problems. Taking the perspective of synthesizing graph theory programs, we propose several extensions to address the issue. First, inspired by the dependency of the iteration number of common graph theory algorithms on graph size, we learn to terminate the message passing process in GNNs adaptively according to the computation progress. Second, inspired by the fact that many graph theory algorithms are homogeneous with respect to…
Sapienza Università di Roma（罗马第一大学）；Istituto Nazionale di Fisica Nucleare（意大利国家核物理研究所）；Institute of Nanotechnology (NANOTEC) - CNR（意大利国家研究委员会纳米技术研究所）
The recent work ``Combinatorial Optimization with Physics-Inspired Graph Neural Networks'' [Nat Mach Intell 4 (2022) 367] introduces a physics-inspired unsupervised Graph Neural Network (GNN) to solve combinatorial optimization problems on sparse graphs. To test the performances of these GNNs, the authors of the work show numerical results for two fundamental problems: maximum cut and maximum independent set (MIS). They conclude that "the graph neural network optimizer performs on par or outperforms existing solvers, with the ability to scale beyond the state of the art to problems with millio…
In Nature Machine Intelligence 4, 367 (2022), Schuetz et al provide a scheme to employ graph neural networks (GNN) as a heuristic to solve a variety of classical, NP-hard combinatorial optimization problems. It describes how the network is trained on sample instances and the resulting GNN heuristic is evaluated applying widely used techniques to determine its ability to succeed. Clearly, the idea of harnessing the powerful abilities of such networks to ``learn'' the intricacies of complex, multimodal energy landscapes in such a hands-off approach seems enticing. And based on the observed perfo…
This work develops neural-network--based preconditioners to accelerate solution of the Wilson-Dirac normal equation in lattice quantum field theories. The approach is implemented for the two-flavor lattice Schwinger model near the critical point. In this system, neural-network preconditioners are found to accelerate the convergence of the conjugate gradient solver compared with the solution of unpreconditioned systems or those preconditioned with conventional approaches based on even-odd or incomplete Cholesky decompositions, as measured by reductions in the number of iterations and/or complex…
Solving systems of linear equations is a problem occuring frequently in water engineering applications. Usually the size of the problem is too large to be solved via direct factorization. One can resort to iterative approaches, in particular the conjugate gradients method if the matrix is symmetric positive definite. Preconditioners further enhance the rate of convergence but hitherto only handcrafted ones requiring expert knowledge have been used. We propose an innovative approach employing Machine Learning, in particular a Convolutional Neural Network, to unassistedly design preconditioning …
In this paper, we present a data-driven approach to iteratively solve the discrete heterogeneous Helmholtz equation at high wavenumbers. In our approach, we combine classical iterative solvers with convolutional neural networks (CNNs) to form a preconditioner which is applied within a Krylov solver. For the preconditioner, we use a CNN of type U-Net that operates in conjunction with multigrid ingredients. Two types of preconditioners are proposed 1) U-Net as a coarse grid solver, and 2) U-Net as a deflation operator with shifted Laplacian V-cycles. Following our training scheme and data-augmen…
