Lie Point Symmetry Data Augmentation for Neural PDE Solvers
University of Amsterdam（阿姆斯特丹大学）；Johannes Kepler University Linz（约翰内斯·开普勒林茨大学）；Microsoft Research（微软研究院）；Qualcomm AI Research（高通人工智能研究院）；Deepmind（深度思维）
Johannes Brandstetter, Max Welling, Daniel E. Worrall
Neural networks are increasingly being used to solve partial differential equations (PDEs), replacing slower numerical solvers. However, a critical issue is that neural PDE solvers require high-quality ground truth data, which usually must come from the very solvers they are designed to replace. Thus, we are presented with a proverbial chicken-and-egg problem. In this paper, we present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity -- Lie point symmetry data augmentation (LPSDA). In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations, based on the Lie point symmetry group of the PDEs in question, something not possible in other application areas. We present this framework and demonstrate how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
Weizmann Institute of Science（魏茨曼科学研究所）；Technion, Israel Institute of Technology（以色列理工学院）
Constructing fast numerical solvers for partial differential equations (PDEs) is crucial for many scientific disciplines. A leading technique for solving large-scale PDEs is using multigrid methods. At the core of a multigrid solver is the prolongation matrix, which relates between different scales of the problem. This matrix is strongly problem-dependent, and its optimal construction is critical to the efficiency of the solver. In practice, however, devising multigrid algorithms for new problems often poses formidable challenges. In this paper we propose a framework for learning multigrid sol…
University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）
Machine learning for scientific applications faces the challenge of limited data. We propose a framework that leverages a priori known physics to reduce overfitting when training on relatively small datasets. A deep neural network is embedded in a partial differential equation (PDE) that expresses the known physics and learns to describe the corresponding unknown or unrepresented physics from the data. Crafted as such, the neural network can also provide corrections for erroneously represented physics, such as discretization errors associated with the PDE's numerical solution. Once trained, th…
In this work we propose a hybrid solver to solve partial differential equation (PDE)s in the latent space. The solver uses an iterative inferencing strategy combined with solution initialization to improve generalization of PDE solutions. The solver is tested on an engineering case and the results show that it can generalize well to several PDE conditions.
One of the main challenges in solving time-dependent partial differential equations is to develop computationally efficient solvers that are accurate and stable. Here, we introduce a graph neural network approach to finding efficient PDE solvers through learning using message-passing models. We first introduce domain invariant features for PDE-data inspired by classical PDE solvers for an efficient physical representation. Next, we use graphs to represent PDE-data on an unstructured mesh and show that message passing graph neural networks (MPGNN) can parameterize governing equations, and as a …
Technical University of Munich（慕尼黑工业大学）；LTCI, Telecom Paris, IP Paris（巴黎理工学院巴黎电信学院LTCI）；Columbia University（哥伦比亚大学）
Finding accurate solutions to partial differential equations (PDEs) is a crucial task in all scientific and engineering disciplines. It has recently been shown that machine learning methods can improve the solution accuracy by correcting for effects not captured by the discretized PDE. We target the problem of reducing numerical errors of iterative PDE solvers and compare different learning approaches for finding complex correction functions. We find that previously used learning approaches are significantly outperformed by methods that integrate the solver into the training loop and thereby a…
Over the last few decades, existing Partial Differential Equation (PDE) solvers have demonstrated a tremendous success in solving complex, non-linear PDEs. Although accurate, these PDE solvers are computationally costly. With the advances in Machine Learning (ML) technologies, there has been a significant increase in the research of using ML to solve PDEs. The goal of this work is to develop an ML-based PDE solver, that couples important characteristics of existing PDE solvers with ML technologies. The two solver characteristics that have been adopted in this work are: 1) the use of discretiza…
Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs
Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；Department of Mathematics, University of Kentucky（肯塔基大学数学系）；Department of Computing and Mathematical Sciences, California Institute of Technology（加州理工学院计算与数学科学系）
We propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs. Our algorithm possesses both temporal and spatial multi-scale resolution features by incorporating multi-scale sequence models and graph gating modules in the encoder and processor, respectively. Benchmark numerical experiments are presented to demonstrate that the proposed algorithm outperforms baselines, particularly on a PDE with a range of spatial and temporal scales.
Fast and accurate solution of time-dependent partial differential equations (PDEs) is of key interest in many research fields including physics, engineering, and biology. Generally, implicit schemes are preferred over the explicit ones for better stability and correctness. The existing implicit schemes are usually iterative and employ a general-purpose solver which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme for a class of PDEs, in a data-driven fashion. We attain this objective by modifying an iteration of an …
Solving partial differential equations is difficult. Recently proposed neural resolution-invariant models, despite their effectiveness and efficiency, usually require equispaced spatial points of data. However, sampling in spatial domain is sometimes inevitably non-equispaced in real-world systems, limiting their applicability. In this paper, we propose a Non-equispaced Fourier PDE Solver (\textsc{NFS}) with adaptive interpolation on resampled equispaced points and a variant of Fourier Neural Operators as its components. Experimental results on complex PDEs demonstrate its advantages in accura…
Promoting Generalization for Exact Solvers via Adversarial Instance Augmentation
CAS Key Laboratory of Technology in GIPAS, University of Science and Technology of China（GIPAS技术CAS重点实验室，中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
University of Stuttgart（斯图加特大学）；SimTech（模拟技术中心）；IMPRS-IS（国际马克斯·普朗克研究学院智能系统研究所）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL University（巴黎文理研究大学）；NEC Labs Europe（日本电气欧洲实验室）
University of Amsterdam（阿姆斯特丹大学）；Johannes Kepler University Linz（林茨约翰内斯·开普勒大学）；Qualcomm AI Research（高通人工智能研究院）
The numerical solution of partial differential equations (PDEs) is difficult, having led to a century of research so far. Recently, there have been pushes to build neural--numerical hybrid solvers, which piggy-backs the modern trend towards fully end-to-end learned systems. Most works so far can only generalize over a subset of properties to which a generic solver would be faced, including: resolution, topology, geometry, boundary conditions, domain discretization regularity, dimensionality, etc. In this work, we build a solver, satisfying these properties, where all the components are based o…
M2NO: An Efficient Multi-Resolution Operator Framework for Dynamic Multi-Scale PDE Solvers
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Southwest Jiaotong University（西南交通大学）
Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers
Department of Engineering Structures（工程结构系）；TU Delft（代尔夫特理工大学）；Department of Electrical Engineering（电气工程系）；TU Eindhoven（埃因霍温理工大学）；Laboratory of Earthquake Engineering（地震工程实验室）；National Technical University of Athens（雅典国家技术大学）；Machine Learning Research Group（机器学习研究组）；University of Oxford（牛津大学）
Machine learned partial differential equation (PDE) solvers trade the reliability of standard numerical methods for potential gains in accuracy and/or speed. The only way for a solver to guarantee that it outputs the exact solution is to use a convergent method in the limit that the grid spacing $Δx$ and timestep $Δt$ approach zero. Machine learned solvers, which learn to update the solution at large $Δx$ and/or $Δt$, can never guarantee perfect accuracy. Some amount of error is inevitable, so the question becomes: how do we constrain machine learned solvers to give us the sorts of errors that…
RICOS Co. Ltd.（RICOS有限公司）；University of Tsukuba（筑波大学）
Graph neural network (GNN) is a promising approach to learning and predicting physical phenomena described in boundary value problems, such as partial differential equations (PDEs) with boundary conditions. However, existing models inadequately treat boundary conditions essential for the reliable prediction of such problems. In addition, because of the locally connected nature of GNNs, it is difficult to accurately predict the state after a long time, where interaction between vertices tends to be global. We present our approach termed physics-embedded neural networks that considers boundary c…
Carnegie Mellon University（卡内基梅隆大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）
Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics. Recent Machine Learning (ML) models trained on low-resolution spatio-temporal signals have shown new promises in capturing important dynamics in high-resolution signals, under the condition that the models can effectively recover the missing details. However, this study shows that significant information is often lost in the low-resolution down-sampled features. To address such issues, we propose a new approac…
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS)（中国科学院数学与系统科学研究院）；Beijing Institute of Technology（北京理工大学）；Zhongguancun Academy（中关村学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solvers
National Center for Applied Mathematics, Tianjin University（天津大学应用数学中心）；School of Mechanical and Aerospace Engineering, Jilin University（吉林大学机械与航空航天工程学院）
UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs
Stony Brook University (SUNY)（纽约州立大学石溪分校）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）
