Physics-Informed Neural Networks (PINNs) have been recognized as a mesh-free alternative to solve partial differential equations where physics information is incorporated. However, in dealing with problems characterized by high stiffness or shock-dominated dynamics, traditional PINNs have been found to have limitations, including unbalanced training and inaccuracy in solution, even with small physics residuals. In this research, we seek to address these limitations using the viscous Burgers' equation with low viscosity and the Allen-Cahn equation as test problems. In addressing unbalanced training, we have developed a new adaptive loss balancing scheme using smoothed gradient norms to ensure satisfaction of initial and boundary conditions. Further, to address inaccuracy in the solution, we have developed an adaptive residual-based collocation scheme to improve the accuracy of solutions in the regions with high physics residuals. The proposed new approach significantly improves solution accuracy with consistent satisfaction of physics residuals. For instance, in the case of Burgers' equation, the relative L2 error is reduced by about 44 percent compared to traditional PINNs, while for the Allen-Cahn equation, the relative L2 error is reduced by approximately 70 percent. Additionally, we show the trustworthy solution comparison of the proposed method using a robust finite difference solver.
Diversity-Aware Adaptive Collocation for Physics-Informed Neural Networks via Sparse QUBO Optimization and Hybrid Coresets
Research Center for Artificial Intelligence（人工智能研究中心）；Innopolis University（因诺波利斯大学）；Moscow Center for Advanced Studies（莫斯科先进研究中心）；Laboratory of Quantum Computing（量子计算实验室）；Center for Engineering Systems and Sciences（工程系统与科学中心）；Central University（中央大学）；The Center for Top-Level Educational Programs in AI（人工智能顶级教育项目中心）；National Medical Research Center of Oncology（肿瘤医学研究中心）
Institute of Informatics（信息学研究所）；Ludwig Maximilians University Munich（慕尼黑路德维希-马克西米利安大学）；NASA Goddard Space Flight Center（美国国家航空航天局戈达德太空飞行中心）
Physics-informed Neural Networks (PINNs) often have, in their loss functions, terms based on physical equations and derivatives. In order to evaluate these terms, the output solution is sampled using a distribution of collocation points. However, density-based strategies, in which the number of collocation points over the domain increases throughout the training period, do not scale well to multiple spatial dimensions. To remedy this issue, we present here a curriculum-training-based method for lightweight collocation point distributions during network training. We apply this method to a PINN …
This paper presents a PINN training framework that employs (1) pre-training steps that accelerates and improve the robustness of the training of physics-informed neural network with auxiliary data stored in point clouds, (2) a net-to-net knowledge transfer algorithm that improves the weight initialization of the neural network and (3) a multi-objective optimization algorithm that may improve the performance of a physical-informed neural network with competing constraints. We consider the training and transfer and multi-task learning of physics-informed neural network (PINN) as multi-objective …
Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；University of Utah（犹他大学）；International Computer Science Institute（国际计算机科学研究所）；University of California Berkeley（加州大学伯克利分校）
Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function, but recent work has shown that this can lead to optimization difficulties. Here, we study the impact of the location of the collocation points on the trainability of these models. We find that the vanilla PINN performance can be significantly boosted by adapting the location of the collocation points as training proceeds. Specifically, we propose a novel adaptive collocation scheme which progressively allocates more collocation points (without increasing th…
The advent of scientific machine learning (SciML) has opened up a new field with many promises and challenges in the field of simulation science by developing approaches at the interface of physics- and data-based modelling. To this end, physics-informed neural networks (PINNs) have been introduced in recent years, which cope for the scarcity in training data by incorporating physics knowledge of the problem at so-called collocation points. In this work, we investigate the prediction performance of PINNs with respect to the number of collocation points used to enforce the physics-based penalty…
Collocation-based Robust Physics Informed Neural Networks for time-dependent simulations of pollution propagation under thermal inversion conditions on Spitsbergen
AGH University of Krakow, Faculty of Computer Science（克拉科夫AGH大学计算机科学系）；Medical University of Silesia-Katowice, Faculty of Medical Sciences（西里西亚-卡托维兹医学院医学科学系）；The University Centre in Svalbard, Longyearbyen（斯瓦尔巴大学长yearbyen分校）；Simula Research Laboratory（Simula研究实验室）；Norwegian University of Life Sciences（挪威生命科学大学）；AGH University of Krakow, Faculty of Energy（克拉科夫AGH大学能源与燃料系）
Physics-Informed Neural Networks (PINNs) have become a kind of attractive machine learning method for obtaining solutions of partial differential equations (PDEs). Training PINNs can be seen as a semi-supervised learning task, in which only exact values of initial and boundary points can be obtained in solving forward problems, and in the whole spatio-temporal domain collocation points are sampled without exact labels, which brings training difficulties. Thus the selection of collocation points and sampling methods are quite crucial in training PINNs. Existing sampling methods include fixed an…
PACMANN: Point Adaptive Collocation Method for Artificial Neural Networks
Faculty of Aerospace Engineering（航空航天工程系）；Delft University of Technology（代尔夫特理工大学）；Delft Institute of Applied Mathematics, Faculty of Electrical Engineering, Mathematics and Computer Science（代尔夫特应用数学研究所，电气工程、数学与计算机科学系）；Delft Institute for Computational Science and Engineering（代尔夫特计算科学与工程研究所）
IBM Research - Europe（IBM欧洲研究院）；The University of Manchester（曼彻斯特大学）；IBM Research（IBM研究院）
We propose a new design of a neural network for solving a zero shot super resolution problem for turbulent flows. We embed Luenberger-type observer into the network's architecture to inform the network of the physics of the process, and to provide error correction and stabilization mechanisms. In addition, to compensate for decrease of observer's performance due to the presence of unknown destabilizing forcing, the network is designed to estimate the contribution of the unknown forcing implicitly from the data over the course of training. By running a set of numerical experiments, we demonstra…
Physics-informed post-processing of stabilized finite element solutions for transient convection-dominated problems
Department of Computer Technologies, Antalya Bilim University（安塔利亚比林大学计算机技术系）；Institute of Applied Mathematics, Middle East Technical University（中东技术大学应用数学研究所）；Department of Mathematics, Indian Institute of Technology Guwahati（印度理工学院古瓦哈提数学系）
Physics-informed Neural Networks (PINNs) have recently emerged as a principled way to include prior physical knowledge in form of partial differential equations (PDEs) into neural networks. Although PINNs are generally viewed as mesh-free, current approaches still rely on collocation points within a bounded region, even in settings with spatially sparse signals. Furthermore, if the boundaries are not known, the selection of such a region is difficult and often results in a large proportion of collocation points being selected in areas of low relevance. To resolve this severe drawback of curren…
Efficient Global-Local Fusion Sampling for Physics-Informed Neural Networks
School of Mathematical Sciences, Soochow University（苏州大学数学科学学院）；Zu Chongzhi Center , Duke Kunshan University（杜克昆山学院钟楚中心）；University of Science and Technology of China（中国科学技术大学）；Key Laboratory of the Ministry of Education for Mathematical Foundations and Applications of Digital Technology（教育部数学基础与应用数字技术重点实验室）
Universidad Adolfo Ibáñez（阿道夫·伊瓦涅斯大学）；Data Observatory Foundation（数据观测站基金会）；Center of Applied Ecology and Sustainability (CAPES)（应用生态与可持续性中心（CAPES））
We consider physics-informed neural networks (PINNs) [Raissi et al., J.~Comput. Phys. 278 (2019) 686-707] for forward physical problems. In order to find optimal PINNs configuration, we introduce a hyper-parameter optimization (HPO) procedure via Gaussian processes-based Bayesian optimization. We apply the HPO to Helmholtz equation for bounded domains and conduct a thorough study, focusing on: (i) performance, (ii) the collocation points density $r$ and (iii) the frequency $κ$, confirming the applicability and necessity of the method. Numerical experiments are performed in two and three dimens…
Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局（A*STAR））；Nanyang Technological University（南洋理工大学）
In this study, novel physics-informed neural network (PINN) methods for coupling neighboring support points and their derivative terms which are obtained by automatic differentiation (AD), are proposed to allow efficient training with improved accuracy. The computation of differential operators required for PINNs loss evaluation at collocation points are conventionally obtained via AD. Although AD has the advantage of being able to compute the exact gradients at any point, such PINNs can only achieve high accuracies with large numbers of collocation points, otherwise they are prone to optimizi…
A Physics-Informed Hierarchical Neural Network for Microwave Scattering Analysis of 3D PEC Targets
Key Laboratory of Universal Wireless Communication, Ministry of Education, School of Information and Communication Engineering, Beijing University of Posts and Telecommunications（信息与通信工程学院，北京邮电大学，教育部无线通信重点实验室）；Department of Informatics and Telecommunications, National and Kapodistrian University of Athens（信息与电信学院，希腊国家与卡波迪斯提亚大学）
Adaptively trained Physics-informed Radial Basis Function Neural Networks for Solving Multi-asset Option Pricing Problems
College of Mathematics and Statistics, Northwest Normal University（西北师范大学数学与统计学院）；Department of Mathematics, City University of Hong Kong（香港城市大学数学系）；Department of Information Technology, Uppsala University（乌普萨拉大学信息科技系）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）
Physics-Informed Neural Networks (PINNs) are a class of deep neural networks that are trained, using automatic differentiation, to compute the response of systems governed by partial differential equations (PDEs). The training of PINNs is simulation-free, and does not require any training dataset to be obtained from numerical PDE solvers. Instead, it only requires the physical problem description, including the governing laws of physics, domain geometry, initial/boundary conditions, and the material properties. This training usually involves solving a non-convex optimization problem using vari…
Physics informed neural networks approximate solutions of PDEs by minimizing pointwise residuals. We derive rigorous bounds on the error, incurred by PINNs in approximating the solutions of a large class of linear parabolic PDEs, namely Kolmogorov equations that include the heat equation and Black-Scholes equation of option pricing, as examples. We construct neural networks, whose PINN residual (generalization error) can be made as small as desired. We also prove that the total $L^2$-error can be bounded by the generalization error, which in turn is bounded in terms of the training error, prov…
Energy Dissipation Rate Guided Adaptive Sampling for Physics-Informed Neural Networks: Resolving Surface-Bulk Dynamics in Allen-Cahn Systems
Department of Mathematics, The Pennsylvania State University（数学系，宾夕法尼亚州立大学）；Department of Mathematics, The Hong Kong University of Science and Technology（数学系，香港科学与技术大学）；Department of Mathematics, University of South Carolina（数学系，南卡罗来纳大学）
Tongji University（同济大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Beijing Institute of Technology（北京理工大学）；King Saud University（沙特国王大学）
In this work, we present a deep collocation method for three dimensional potential problems in nonhomogeneous media. This approach utilizes a physics informed neural network with material transfer learning reducing the solution of the nonhomogeneous partial differential equations to an optimization problem. We tested different cofigurations of the physics informed neural network including smooth activation functions, sampling methods for collocation points generation and combined optimizers. A material transfer learning technique is utilised for nonhomogeneous media with different material gra…
Minzu University of China（中央民族大学）；North China University of Technology（北方工业大学）
As a typical application of deep learning, physics-informed neural network (PINN) {has been} successfully used to find numerical solutions of partial differential equations (PDEs), but how to improve the limited accuracy is still a great challenge for PINN. In this work, we introduce a new method, symmetry-enhanced physics informed neural network (SPINN) where the invariant surface conditions induced by the Lie symmetries or non-classical symmetries of PDEs are embedded into the loss function in PINN, to improve the accuracy of PINN for solving the forward and inverse problems of PDEs. We test…
SPARC-Net: A Spectral, Causality-Aware, and Hard-Constrained Physics-Informed Architecture for Stiff and Shock-Dominated Partial Differential Equations
University of Naples Federico II（那不勒斯费德里科二世大学）；SISSA – International School for Advanced Studies（国际高等研究学院（SISSA））；University of Colorado Boulder（科罗拉多大学博尔德分校）
Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related adv…
CI-PINN: Causal Integral Physics-Informed Neural Network for Solving Evolution Equations
Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Beijing Normal University（北京师范大学）；Guangzhou Nanfang College（广州南方学院）；Louisiana State University（路易斯安那州立大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
Leibniz Universität Hannover（莱布尼茨汉诺威大学）；Tongji University（同济大学）；Ton Duc Thang University（孙德胜大学）
In this work, a modified neural architecture search method (NAS) based physics-informed deep learning model is presented for stochastic analysis in heterogeneous porous material. Monte Carlo method based on a randomized spectral representation is first employed to construct a stochastic model for simulation of flow through porous media. To solve the governing equations for stochastic groundwater flow problem, we build a modified NAS model based on physics-informed neural networks (PINNs) with transfer learning in this paper that will be able to fit different partial differential equations (PDE…
Drug Release Modeling using Physics-Informed Neural Networks
Division of Applied Mathematics, Brown University（应用数学系，布朗大学）；School of Engineering, Brown University（工程学院，布朗大学）；Institute for Biology, Engineering and Medicine, Brown University（生物、工程与医学研究所，布朗大学）
On the training of physics-informed neural operators for solving parametric partial differential equations
College of Civil Engineering, Tongji University（同济大学土木工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Institute for Foundations of Data Science, Yale University（耶鲁大学数据科学基础研究所）
