Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correction
Institute of High Performance Computing（高性能计算研究所）；Centre for Frontier AI Research（前沿人工智能研究中心）；School of Mechanical Engineering（机械工程学院）；College of Computing and Data Science（计算与数据科学学院）
Pao-Hsiung Chiu, Jian Cheng Wong, Chin Chun Ooi, Chang Wei, Yuchen Fan, Yew-Soon Ong
Physics-informed neural networks (PINNs) have emerged as a promising mesh-free paradigm for solving partial differential equations, yet adoption in science and engineering is limited by slow training and modest accuracy relative to modern numerical solvers. We introduce the Sequential Correction Algorithm for Learning Efficient PINN (Scale-PINN), a learning strategy that bridges modern physics-informed learning with numerical algorithms. Scale-PINN incorporates the iterative residual-correction principle, a cornerstone of numerical solvers, directly into the loss formulation, marking a paradigm shift in how PINN losses can be conceived and constructed. This integration enables Scale-PINN to achieve unprecedented convergence speed across PDE problems from different physics domain, including reducing training time on a challenging fluid-dynamics problem for state-of-the-art PINN from hours to sub-2 minutes while maintaining superior accuracy, and enabling application to representative problems in aerodynamics and urban science. By uniting the rigor of numerical methods with the flexibility of deep learning, Scale-PINN marks a significant leap toward the practical adoption of PINNs in science and engineering through scalable, physics-informed learning. Codes are available at https://github.com/chiuph/SCALE-PINN.
Physics-informed neural networks (PINNs) as a means of discretizing partial differential equations (PDEs) are garnering much attention in the Computational Science and Engineering (CS&E) world. At least two challenges exist for PINNs at present: an understanding of accuracy and convergence characteristics with respect to tunable parameters and identification of optimization strategies that make PINNs as efficient as other computational science tools. The cost of PINNs training remains a major challenge of Physics-informed Machine Learning (PiML) - and, in fact, machine learning (ML) in general…
Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientific problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but a…
From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning
Brown University（布朗大学）；Division of Applied Mathematics, Brown University（布朗大学应用数学系）；School of Engineering, Brown University（布朗大学工程学院）；Center for Biomedical Engineering, Brown University（布朗大学生物医学工程中心）
This paper presents NSGA-PINN, a multi-objective optimization framework for effective training of Physics-Informed Neural Networks (PINNs). The proposed framework uses the Non-dominated Sorting Genetic Algorithm (NSGA-II) to enable traditional stochastic gradient optimization algorithms (e.g., ADAM) to escape local minima effectively. Additionally, the NSGA-II algorithm enables satisfying the initial and boundary conditions encoded into the loss function during physics-informed training precisely. We demonstrate the effectiveness of our framework by applying NSGA-PINN to several ordinary and p…
Technical University of Munich（慕尼黑工业大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；Harvard University（哈佛大学）
Physics-Informed Neural Networks (PINNs) are gaining popularity as a method for solving differential equations. While being more feasible in some contexts than the classical numerical techniques, PINNs still lack credibility. A remedy for that can be found in Uncertainty Quantification (UQ) which is just beginning to emerge in the context of PINNs. Assessing how well the trained PINN complies with imposed differential equation is the key to tackling uncertainty, yet there is lack of comprehensive methodology for this task. We propose a framework for UQ in Bayesian PINNs (B-PINNs) that incorpor…
Dalian University of Technology（大连理工大学）；Harbin Institute of Technology（哈尔滨工业大学）；JD Explore Academy（京东探索研究院）
Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs), and have been widely used in a variety of PDE problems. However, there still remain some challenges in the application of PINNs: 1) the mechanism of PINNs is unsuitable (at least cannot be directly applied) to exploiting a small size of (usually very few) extra informative samples to refine the networks; and 2) the efficiency of training PINNs often becomes low for some complicated PDEs. In this paper, we propose the generative adversarial physics-informed n…
School of Data Science（数据科学学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Department of Computer Science（计算机科学系）；Northwestern University（西北大学）；School of Science, Great Bay University（大湾大学科学学院）；Department of Mathematics（数学系）
Physics-Informed Neural Networks (PINNs) have recently been proposed to solve scientific and engineering problems, where physical laws are introduced into neural networks as prior knowledge. With the embedded physical laws, PINNs enable the estimation of critical parameters, which are unobservable via physical tools, through observable variables. For example, Power Electronic Converters (PECs) are essential building blocks for the green energy transition. PINNs have been applied to estimate the capacitance, which is unobservable during PEC operations, using current and voltage, which can be ob…
Multifidelity simulation methodologies are often used in an attempt to judiciously combine low-fidelity and high-fidelity simulation results in an accuracy-increasing, cost-saving way. Candidates for this approach are simulation methodologies for which there are fidelity differences connected with significant computational cost differences. Physics-informed Neural Networks (PINNs) are candidates for these types of approaches due to the significant difference in training times required when different fidelities (expressed in terms of architecture width and depth as well as optimization criteria…
Conformal mapping based Physics-informed neural networks for designing neutral inclusions
Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology（韩国科学技术院数学科学系）；Department of Artificial Intelligence, Chung-Ang University（Chung-Ang 大学人工智能系）
Data Science and Analytics, The Hong Kong University of Science and Technology (Guangzhou)（数据科学与分析，香港科学与技术大学（广州））；Internet of Things, The Hong Kong University of Science and Technology (Guangzhou)（物联网，香港科学与技术大学（广州））；Department of CEE, The Hong Kong University of Science and Technology（工程学院，香港科学与技术大学）
Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles
Departamento de Ciencia de la Computación, Pontificia Universidad Católica de Chile（计算机科学系，天主教智利大学）；Department of Computer Science, University of Tübingen（计算机科学系，图宾根大学）；John A. Paulson School of Engineering and Applied Sciences, Harvard University（工程与应用科学学院，哈佛大学）
Brown University（布朗大学）；Xiamen University（厦门大学）；Dalian University of Technology（大连理工大学）
Despite the significant progress over the last 50 years in simulating flow problems using numerical discretization of the Navier-Stokes equations (NSE), we still cannot incorporate seamlessly noisy data into existing algorithms, mesh-generation is complex, and we cannot tackle high-dimensional problems governed by parametrized NSE. Moreover, solving inverse flow problems is often prohibitively expensive and requires complex and expensive formulations and new computer codes. Here, we review flow physics-informed learning, integrating seamlessly data and mathematical models, and implementing the…
Physics-Informed Neural Networks (PINNs) offer a promising approach to solving differential equations and, more generally, to applying deep learning to problems in the physical sciences. We adopt a recently developed transfer learning approach for PINNs and introduce a multi-head model to efficiently obtain accurate solutions to nonlinear systems of ordinary differential equations with random potentials. In particular, we apply the method to simulate stochastic branched flows, a universal phenomenon in random wave dynamics. Finally, we compare the results achieved by feed forward and GAN-based…
Physics-informed neural networks (PINNs) leverage neural-networks to find the solutions of partial differential equation (PDE)-constrained optimization problems with initial conditions and boundary conditions as soft constraints. These soft constraints are often considered to be the sources of the complexity in the training phase of PINNs. Here, we demonstrate that the challenge of training (i) persists even when the boundary conditions are strictly enforced, and (ii) is closely related to the Kolmogorov n-width associated with problems demonstrating transport, convection, traveling waves, or …
Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
Agency for Science, Technology and Research (A*STAR)（科技研究局）；Indian Institute of Technology (IIT) Goa（印度理工学院 Goa分校）；Nanyang Technological University（南洋理工大学）
Physics-Informed Neural Network with Squeeze-Excitation-like Attention
Key Laboratory of Quark and Lepton Physics (MOE) & Institute of Particle Physics, Central China Normal University（华中师范大学夸克与轻子物理教育部重点实验室及粒子物理研究所）；Artificial Intelligence and Computational Physics Research Center, Central China Normal University（华中师范大学人工智能与计算物理研究中心）；Key Laboratory of Nuclear Physics and Ion-beam Application (MOE) & Institute of Modern Physics, Fudan University（复旦大学核物理与离子束应用教育部重点实验室及现代物理研究所）；Shanghai Research Center for Theoretical Nuclear Physics, NSFC and Fudan University（国家自然科学基金委员会-复旦大学上海理论核物理研究中心）；Northwest Institute of Nuclear Technology（西北核技术研究所）
Physics informed neural networks approximate solutions of PDEs by minimizing pointwise residuals. We derive rigorous bounds on the error, incurred by PINNs in approximating the solutions of a large class of linear parabolic PDEs, namely Kolmogorov equations that include the heat equation and Black-Scholes equation of option pricing, as examples. We construct neural networks, whose PINN residual (generalization error) can be made as small as desired. We also prove that the total $L^2$-error can be bounded by the generalization error, which in turn is bounded in terms of the training error, prov…
Deep learning-based surrogate modeling is becoming a promising approach for learning and simulating dynamical systems. Deep-learning methods, however, find very challenging learning stiff dynamics. In this paper, we develop DAE-PINN, the first effective deep-learning framework for learning and simulating the solution trajectories of nonlinear differential-algebraic equations (DAE), which present a form of infinite stiffness and describe, for example, the dynamics of power networks. Our DAE-PINN bases its effectiveness on the synergy between implicit Runge-Kutta time-stepping schemes (designed …
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Department of Electronics and Computer Science（电子与计算机科学系）；Mondragon University（蒙得龙大学）；Research & Development（研发）；CARTIF Technology Centre（CARTIF技术中心）；Department of Physical Chemistry（物理化学系）；University of the Basque Country (EHU)（巴斯克国家大学（EHU））；Basque Centre for Applied Mathematics (BCAM)（巴斯克应用数学中心（BCAM））；Department of Computer Science and Artificial Intelligence（计算机科学与人工智能系）
National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Physics-Informed Neural Networks (PINNs) can be regarded as general-purpose PDE solvers, but it might be slow to train PINNs on particular problems, and there is no theoretical guarantee of corresponding error bounds. In this manuscript, we propose a variant called Prior Dictionary based Physics-Informed Neural Networks (PD-PINNs). Equipped with task-dependent dictionaries, PD-PINNs enjoy enhanced representation power on the tasks, which helps to capture features provided by dictionaries so that the proposed neural networks can achieve faster convergence in the process of training. In various …
St. Mark’s School of Texas（圣马克德克萨斯学校）；University of Pennsylvania（宾夕法尼亚大学）；Brown University（布朗大学）
Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy and training efficiency of PINNs.…
Physics-Informed Neural Networks with Fourier Features and Attention-Driven Decoding
High Technology High School（高科技高中）；Institute for Gravitation and the Cosmos（引力与宇宙研究所）；The Pennsylvania State University（宾夕法尼亚州立大学）；Department of Physics（物理系）；Princeton University（普林斯顿大学）
Numerical solutions to the equation for advection are determined using different finite-difference approximations and physics-informed neural networks (PINNs) under conditions that allow an analytical solution. Their accuracy is examined by comparing them to the analytical solution. We used a machine learning framework like PyTorch to implement PINNs. PINNs approach allows training neural networks while respecting the PDEs as a strong constraint in the optimization as apposed to making them part of the loss function. In standard small-scale circulation simulations, it is shown that the convent…
Adaptive Interface-PINNs (AdaI-PINNs): An Efficient Physics-informed Neural Networks Framework for Interface Problems
Department of Civil Engineering, Indian Institute of Technology, Madras, Chennai 600036, Tamil Nadu, India（印度理工学院马德拉斯分校土木工程系）；Atmospheric, Earth and Energy Division, Lawrence Livermore National Laboratory, Livermore, California 94551, USA（劳伦斯利弗莫尔国家实验室大气、地球与能源部）；Institute of Civil Engineering, École Polytechnique Fédérale de Lausanne, Station 18-1015, Lausanne, Switzerland（洛桑联邦理工学院土木工程学院）
Physics-informed neural networks (PINNs) have recently received much attention due to their capabilities in solving both forward and inverse problems. For training a deep neural network associated with a PINN, one typically constructs a total loss function using a weighted sum of different loss terms and then tries to minimize that. This approach often becomes problematic for solving stiff equations since it cannot consider adaptive increments. Many studies reported the poor performance of the PINN and its challenges in simulating stiff chemical active issues with administering conditions of s…
xLSTM-PINN: Memory-Gated Spectral Remodeling for Physics-Informed Learning
Nanophotonics and Biophotonics Key Laboratory of Jilin Province, School of Physics, Changchun University of Science and Technology（吉林省纳米光子与生物光子重点实验室，物理学院，长春理工大学）
FO-PINNs: A First-Order formulation for Physics Informed Neural Networks
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）；University of California Davis（加州大学戴维斯分校）；Illinois Institute of Technology（伊利诺伊理工学院）
