用于偏微分方程（PDEs）的神经网络求解器已取得显著进展，但仍面临数据稀缺和模型鲁棒性方面的挑战。传统的数据增强方法利用对称性或不变性，对物理系统施加了强假设，而这些假设在动态且复杂的现实应用中往往不成立。为弥补这一研究空白，本研究提出了一种面向神经网络 PDEs 的通用学习策略，名为 Systematic Model Augmentation for Robust Training（SMART）。SMART 通过聚焦于挑战并改进模型的薄弱环节，在数据稀缺条件下降低训练过程中的泛化误差，从而在多种 PDE 场景中显著提升预测精度。所提方法的有效性通过理论分析和大量实验得到了验证。代码将公开提供。
英文摘要：
Neural network solvers for partial differential equations (PDEs) have made significant progress, yet they continue to face challenges related to data scarcity and model robustness. Traditional data augmentation methods, which leverage symmetry or invariance, impose strong assumptions on physical systems that often do not hold in dynamic and complex real-world applications. To address this research gap, this study introduces a universal learning strategy for neural network PDEs, named Systematic Model Augmentation for Robust Training (SMART). By focusing on challenging and improving the model's weaknesses, SMART reduces generalization error during training under data-scarce conditions, leading to significant improvements in prediction accuracy across various PDE scenarios. The effectiveness of the proposed method is demonstrated through both theoretical analysis and extensive experimentation. The code will be available.
University of Amsterdam（阿姆斯特丹大学）；Johannes Kepler University Linz（约翰内斯·开普勒林茨大学）；Microsoft Research（微软研究院）；Qualcomm AI Research（高通人工智能研究院）；Deepmind（深度思维）
Neural networks are increasingly being used to solve partial differential equations (PDEs), replacing slower numerical solvers. However, a critical issue is that neural PDE solvers require high-quality ground truth data, which usually must come from the very solvers they are designed to replace. Thus, we are presented with a proverbial chicken-and-egg problem. In this paper, we present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity -- Lie point symmetry data augmentation (LPSDA). In the context of PDEs, it turns out that we are able to qu…
Weizmann Institute of Science（魏茨曼科学研究所）；Technion, Israel Institute of Technology（以色列理工学院）
Constructing fast numerical solvers for partial differential equations (PDEs) is crucial for many scientific disciplines. A leading technique for solving large-scale PDEs is using multigrid methods. At the core of a multigrid solver is the prolongation matrix, which relates between different scales of the problem. This matrix is strongly problem-dependent, and its optimal construction is critical to the efficiency of the solver. In practice, however, devising multigrid algorithms for new problems often poses formidable challenges. In this paper we propose a framework for learning multigrid sol…
In this work we propose a hybrid solver to solve partial differential equation (PDE)s in the latent space. The solver uses an iterative inferencing strategy combined with solution initialization to improve generalization of PDE solutions. The solver is tested on an engineering case and the results show that it can generalize well to several PDE conditions.
One of the main challenges in solving time-dependent partial differential equations is to develop computationally efficient solvers that are accurate and stable. Here, we introduce a graph neural network approach to finding efficient PDE solvers through learning using message-passing models. We first introduce domain invariant features for PDE-data inspired by classical PDE solvers for an efficient physical representation. Next, we use graphs to represent PDE-data on an unstructured mesh and show that message passing graph neural networks (MPGNN) can parameterize governing equations, and as a …
Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs
Kahlert School of Computing, University of Utah（犹他大学卡勒特计算学院）；Department of Mathematics, University of Kentucky（肯塔基大学数学系）；Department of Computing and Mathematical Sciences, California Institute of Technology（加州理工学院计算与数学科学系）
Solving partial differential equations is difficult. Recently proposed neural resolution-invariant models, despite their effectiveness and efficiency, usually require equispaced spatial points of data. However, sampling in spatial domain is sometimes inevitably non-equispaced in real-world systems, limiting their applicability. In this paper, we propose a Non-equispaced Fourier PDE Solver (\textsc{NFS}) with adaptive interpolation on resampled equispaced points and a variant of Fourier Neural Operators as its components. Experimental results on complex PDEs demonstrate its advantages in accura…
M2NO: An Efficient Multi-Resolution Operator Framework for Dynamic Multi-Scale PDE Solvers
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Southwest Jiaotong University（西南交通大学）
Technical University of Munich（慕尼黑工业大学）；LTCI, Telecom Paris, IP Paris（巴黎理工学院巴黎电信学院LTCI）；Columbia University（哥伦比亚大学）
Finding accurate solutions to partial differential equations (PDEs) is a crucial task in all scientific and engineering disciplines. It has recently been shown that machine learning methods can improve the solution accuracy by correcting for effects not captured by the discretized PDE. We target the problem of reducing numerical errors of iterative PDE solvers and compare different learning approaches for finding complex correction functions. We find that previously used learning approaches are significantly outperformed by methods that integrate the solver into the training loop and thereby a…
Carnegie Mellon University（卡内基梅隆大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）
Numerical simulation of non-linear partial differential equations plays a crucial role in modeling physical science and engineering phenomena, such as weather, climate, and aerodynamics. Recent Machine Learning (ML) models trained on low-resolution spatio-temporal signals have shown new promises in capturing important dynamics in high-resolution signals, under the condition that the models can effectively recover the missing details. However, this study shows that significant information is often lost in the low-resolution down-sampled features. To address such issues, we propose a new approac…
Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solvers
National Center for Applied Mathematics, Tianjin University（天津大学应用数学中心）；School of Mechanical and Aerospace Engineering, Jilin University（吉林大学机械与航空航天工程学院）
Over the last few decades, existing Partial Differential Equation (PDE) solvers have demonstrated a tremendous success in solving complex, non-linear PDEs. Although accurate, these PDE solvers are computationally costly. With the advances in Machine Learning (ML) technologies, there has been a significant increase in the research of using ML to solve PDEs. The goal of this work is to develop an ML-based PDE solver, that couples important characteristics of existing PDE solvers with ML technologies. The two solver characteristics that have been adopted in this work are: 1) the use of discretiza…
Fast and accurate solution of time-dependent partial differential equations (PDEs) is of key interest in many research fields including physics, engineering, and biology. Generally, implicit schemes are preferred over the explicit ones for better stability and correctness. The existing implicit schemes are usually iterative and employ a general-purpose solver which may be sub-optimal for a specific class of PDEs. In this paper, we propose a neural solver to learn an optimal iterative scheme for a class of PDEs, in a data-driven fashion. We attain this objective by modifying an iteration of an …
We propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs. Our algorithm possesses both temporal and spatial multi-scale resolution features by incorporating multi-scale sequence models and graph gating modules in the encoder and processor, respectively. Benchmark numerical experiments are presented to demonstrate that the proposed algorithm outperforms baselines, particularly on a PDE with a range of spatial and temporal scales.
Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers
Department of Engineering Structures（工程结构系）；TU Delft（代尔夫特理工大学）；Department of Electrical Engineering（电气工程系）；TU Eindhoven（埃因霍温理工大学）；Laboratory of Earthquake Engineering（地震工程实验室）；National Technical University of Athens（雅典国家技术大学）；Machine Learning Research Group（机器学习研究组）；University of Oxford（牛津大学）
RICOS Co. Ltd.（RICOS有限公司）；University of Tsukuba（筑波大学）
Graph neural network (GNN) is a promising approach to learning and predicting physical phenomena described in boundary value problems, such as partial differential equations (PDEs) with boundary conditions. However, existing models inadequately treat boundary conditions essential for the reliable prediction of such problems. In addition, because of the locally connected nature of GNNs, it is difficult to accurately predict the state after a long time, where interaction between vertices tends to be global. We present our approach termed physics-embedded neural networks that considers boundary c…
University of Stuttgart（斯图加特大学）；SimTech（模拟技术中心）；IMPRS-IS（国际马克斯·普朗克研究学院智能系统研究所）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL University（巴黎文理研究大学）；NEC Labs Europe（日本电气欧洲实验室）
Machine learned partial differential equation (PDE) solvers trade the reliability of standard numerical methods for potential gains in accuracy and/or speed. The only way for a solver to guarantee that it outputs the exact solution is to use a convergent method in the limit that the grid spacing $Δx$ and timestep $Δt$ approach zero. Machine learned solvers, which learn to update the solution at large $Δx$ and/or $Δt$, can never guarantee perfect accuracy. Some amount of error is inevitable, so the question becomes: how do we constrain machine learned solvers to give us the sorts of errors that…
Center for Combustion Energy, Tsinghua University（清华大学燃烧能源中心）；College of AI, Tsinghua University（清华大学人工智能学院）；Department of Mechanical Engineering, University College London（伦敦大学学院机械工程系）
Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS)（中国科学院数学与系统科学研究院）；Beijing Institute of Technology（北京理工大学）；Zhongguancun Academy（中关村学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs
Stony Brook University (SUNY)（纽约州立大学石溪分校）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）
