Physics-informed neural networks (PINNs) have shown promise in solving various partial differential equations (PDEs). However, training pathologies have negatively affected the convergence and prediction accuracy of PINNs, which further limits their practical applications. In this paper, we propose to use condition number as a metric to diagnose and mitigate the pathologies in PINNs. Inspired by classical numerical analysis, where the condition number measures sensitivity and stability, we highlight its pivotal role in the training dynamics of PINNs. We prove theorems to reveal how condition number is related to both the error control and convergence of PINNs. Subsequently, we present an algorithm that leverages preconditioning to improve the condition number. Evaluations of 18 PDE problems showcase the superior performance of our method. Significantly, in 7 of these problems, our method reduces errors by an order of magnitude. These empirical findings verify the critical role of the condition number in PINNs' training.
Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
Division of Applied Mathematics, Brown University（布朗大学应用数学系）；Euler Institute, Università della Svizzera italiana（瑞士意大利语区大学欧拉研究所）；UniDistance, Switzerland（瑞士远程大学）
Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization
Dept. of Control and Instrumentation Engineering, Korea University, Sejong, South Korea（控制与仪器工程系，韩国大学，世宗，韩国）；BK21 FOUR Smart Mobility Education and Research Team, Korea University, Sejong, South Korea（BK21 FOUR智能交通教育与研究团队，韩国大学，世宗，韩国）
Trainability-Oriented Hybrid Quantum Regression via Geometric Preconditioning and Curriculum Optimization
Shanghai Jiao Tong University-Chongqing Institute of Artificial Intelligence（上海交通大学-重庆人工智能研究院）；Department of Mathematics, National University of Singapore（新加坡国立大学数学系）
ETH Zürich（苏黎世联邦理工学院）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. These bounds guarantee that PINNs and (physics-informed) DeepONets or FNOs will efficiently approximate the underlying solution or solution operator of generic partial differential equations (PDEs). Our framework utilizes existing neural network approximation results to obtain bounds on more involved learning architectures for PDEs. We illustrate …
The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The $L^2$ Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With…
University of Twente（特文特大学）；Massachusetts Institute of Technology（麻省理工学院）
An energy-based a posteriori error bound is proposed for the physics-informed neural network solutions of elasticity problems. An admissible displacement-stress solution pair is obtained from a mixed form of physics-informed neural networks, and the proposed error bound is formulated as the constitutive relation error defined by the solution pair. Such an error estimator provides an upper bound of the global error of neural network discretization. The bounding property, as well as the asymptotic behavior of the physics-informed neural network solutions, are studied in a demonstrating example.
We prove rigorous bounds on the errors resulting from the approximation of the incompressible Navier-Stokes equations with (extended) physics informed neural networks. We show that the underlying PDE residual can be made arbitrarily small for tanh neural networks with two hidden layers. Moreover, the total error can be estimated in terms of the training error, network size and number of quadrature points. The theory is illustrated with numerical experiments.
University of Nevada, Las Vegas（内华达大学拉斯维加斯分校）；University of Arkansas（阿肯色大学）；Lehigh University（里海大学）
In this paper, we show a physics-informed neural network solver for the time-dependent surface PDEs. Unlike the traditional numerical solver, no extension of PDE and mesh on the surface is needed. We show a simplified prior estimate of the surface differential operators so that PINN's loss value will be an indicator of the residue of the surface PDEs. Numerical experiments verify efficacy of our algorithm.
An operator preconditioning perspective on training in physics-informed machine learning
ETH Zürich（苏黎世联邦理工学院）；Sorbonne Université（索邦大学）；Extrality（埃克斯特拉蒂（Extrality））；ETH AI Center（苏黎世联邦理工学院人工智能中心）；Seminar for Applied Mathematics（应用数学研讨班）；Institute of Intelligent Systems and Robotics（智能系统与机器人研究所）
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
Technical University of Denmark（丹麦技术大学）；The Danish Hydrocarbon Research and Technology Centre（丹麦碳氢化合物研究与技术中心）
This paper presents the potential of applying physics-informed neural networks for solving nonlinear multiphysics problems, which are essential to many fields such as biomedical engineering, earthquake prediction, and underground energy harvesting. Specifically, we investigate how to extend the methodology of physics-informed neural networks to solve both the forward and inverse problems in relation to the nonlinear diffusivity and Biot's equations. We explore the accuracy of the physics-informed neural networks with different training example sizes and choices of hyperparameters. The impacts …
Xi'an University of Architecture and Technology（西安建筑科技大学）；Xidian University（西安电子科技大学）
Big-data-based artificial intelligence (AI) supports profound evolution in almost all of science and technology. However, modeling and forecasting multi-physical systems remain a challenge due to unavoidable data scarcity and noise. Improving the generalization ability of neural networks by "teaching" domain knowledge and developing a new generation of models combined with the physical laws have become promising areas of machine learning research. Different from "deep" fully-connected neural networks embedded with physical information (PINN), a novel shallow framework named physics-informed co…
Partition of Unity Physics-Informed Neural Networks (POU-PINNs): An Unsupervised Framework for Physics-Informed Domain Decomposition and Mixtures of Experts
The University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Clarkson University（克拉克森大学）；Texas A&M University in Kingsville（德克萨斯农工大学金斯维尔分校）
This paper presents NSGA-PINN, a multi-objective optimization framework for effective training of Physics-Informed Neural Networks (PINNs). The proposed framework uses the Non-dominated Sorting Genetic Algorithm (NSGA-II) to enable traditional stochastic gradient optimization algorithms (e.g., ADAM) to escape local minima effectively. Additionally, the NSGA-II algorithm enables satisfying the initial and boundary conditions encoded into the loss function during physics-informed training precisely. We demonstrate the effectiveness of our framework by applying NSGA-PINN to several ordinary and p…
PIDT: Physics-Informed Digital Twin for Optical Fiber Parameter Estimation
Dept. of Electrical Engineering, Chalmers Univ. of Technology, Sweden（电气工程系，瑞典查尔姆斯理工大学）；Dept. of Microtechnology and Nanoscience, Chalmers Univ. of Technology, Sweden（微电子与纳米科技系，瑞典查尔姆斯理工大学）
Brown University（布朗大学）；Xiamen University（厦门大学）；Dalian University of Technology（大连理工大学）
Despite the significant progress over the last 50 years in simulating flow problems using numerical discretization of the Navier-Stokes equations (NSE), we still cannot incorporate seamlessly noisy data into existing algorithms, mesh-generation is complex, and we cannot tackle high-dimensional problems governed by parametrized NSE. Moreover, solving inverse flow problems is often prohibitively expensive and requires complex and expensive formulations and new computer codes. Here, we review flow physics-informed learning, integrating seamlessly data and mathematical models, and implementing the…
We explore the possibility to use physics-informed neural networks to drastically accelerate the solution of ordinary differential-algebraic equations that govern the power system dynamics. When it comes to transient stability assessment, the traditionally applied methods either carry a significant computational burden, require model simplifications, or use overly conservative surrogate models. Conventional neural networks can circumvent these limitations but are faced with high demand of high-quality training datasets, while they ignore the underlying governing equations. Physics-informed neu…
Bayesian Physics Informed Neural Networks (B-PINNs) have gained significant attention for inferring physical parameters and learning the forward solutions for problems based on partial differential equations. However, the overparameterized nature of neural networks poses a computational challenge for high-dimensional posterior inference. Existing inference approaches, such as particle-based or variance inference methods, are either computationally expensive for high-dimensional posterior inference or provide unsatisfactory uncertainty estimates. In this paper, we present a new efficient infere…
Physics-Informed Neural Networks (PINNs) offer a promising approach to solving differential equations and, more generally, to applying deep learning to problems in the physical sciences. We adopt a recently developed transfer learning approach for PINNs and introduce a multi-head model to efficiently obtain accurate solutions to nonlinear systems of ordinary differential equations with random potentials. In particular, we apply the method to simulate stochastic branched flows, a universal phenomenon in random wave dynamics. Finally, we compare the results achieved by feed forward and GAN-based…
Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of North Carolina, Charlotte（北卡罗来纳大学夏洛特分校）；Delft University of Technology（代尔夫特理工大学）；University of Washington, Applied Mathematics（华盛顿大学应用数学系）；Brown University, Applied Mathematics（布朗大学应用数学系）
Conformal mapping based Physics-informed neural networks for designing neutral inclusions
Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology（韩国科学技术院数学科学系）；Department of Artificial Intelligence, Chung-Ang University（Chung-Ang 大学人工智能系）
The objective of this paper is to investigate the ability of physics-informed neural networks to learn the magnetic field response as a function of design parameters in the context of a two-dimensional (2-D) magnetostatic problem. Our approach is as follows. First, we present a functional whose minimization is equivalent to solving parametric magnetostatic problems. Subsequently, we use a deep neural network (DNN) to represent the magnetic field as a function of space and parameters that describe geometric features and operating points. We train the DNN by minimizing the physics-informed funct…
Physics-informed deep learning has drawn tremendous interest in recent years to solve computational physics problems, whose basic concept is to embed physical laws to constrain/inform neural networks, with the need of less data for training a reliable model. This can be achieved by incorporating the residual of physics equations into the loss function. Through minimizing the loss function, the network could approximate the solution. In this paper, we propose a mixed-variable scheme of physics-informed neural network (PINN) for fluid dynamics and apply it to simulate steady and transient lamina…
Technical University of Denmark（丹麦技术大学）；Center for Electric Power and Energy（电力与能源中心）
This paper introduces for the first time, to our knowledge, a framework for physics-informed neural networks in power system applications. Exploiting the underlying physical laws governing power systems, and inspired by recent developments in the field of machine learning, this paper proposes a neural network training procedure that can make use of the wide range of mathematical models describing power system behavior, both in steady-state and in dynamics. Physics-informed neural networks require substantially less training data and can result in simpler neural network structures, while achiev…
Massachusetts Institute of Technology（麻省理工学院）；Sandia National Laboratories（桑迪亚国家实验室）
Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networ…
