Recent advances of data-driven machine learning have revolutionized fields like computer vision, reinforcement learning, and many scientific and engineering domains. In many real-world and scientific problems, systems that generate data are governed by physical laws. Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data, which makes the intersection of machine learning and physics become a prevailing paradigm. By integrating the data and mathematical physics models seamlessly, it can guide the machine learning model towards solutions that are physically plausible, improving accuracy and efficiency even in uncertain and high-dimensional contexts. In this survey, we present this learning paradigm called Physics-Informed Machine Learning (PIML) which is to build a model that leverages empirical data and available physical prior knowledge to improve performance on a set of tasks that involve a physical mechanism. We systematically review the recent development of physics-informed machine learning from three perspectives of machine learning tasks, representation of physical prior, and methods for incorporating physical prior. We also propose several important open research problems based on the current trends in the field. We argue that encoding different forms of physical prior into model architectures, optimizers, inference algorithms, and significant domain-specific applications like inverse engineering design and robotic control is far from being fully explored in the field of physics-informed machine learning. We believe that the interdisciplinary research of physics-informed machine learning will significantly propel research progress, foster the creation of more effective machine learning models, and also offer invaluable assistance in addressing long-standing problems in related disciplines.
Reinforcement Learning has shown success in a number of complex virtual environments. However, many challenges still exist towards solving problems with natural language as a core component. Interactive Fiction Games (or Text Games) are one such problem type that offer a set of partially observable environments where natural language is required as part of the reinforcement learning solutions. Therefore, this survey's aim is to assist in the development of new Text Game problem settings and solutions for Reinforcement Learning informed by natural language. Specifically, this survey summarises:…
University of Oxford（牛津大学）；Facebook AI Research（Facebook人工智能研究院）；Massachusetts Institute of Technology（麻省理工学院）；University College London（伦敦大学学院）
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge from text corpora and integrate this knowledge into downstream decision making problems. We thus argue that the time is right to investigate a tight integration of natural language understanding into RL in particular. We survey the state of the field, including work on instruction…
C-STEP: Continuous Space-Time Empowerment for Physics-informed Safe Reinforcement Learning of Mobile Agents
Faculty of Technology, Bielefeld University, Germany（比勒菲尔德大学技术学院，德国）；Department of Electrical Engineering and Information Technology, Paderborn University, Germany（帕德博恩大学电气工程与信息科技系，德国）
Physics-informed machine learning (PIML), referring to the combination of prior knowledge of physics, which is the high level abstraction of natural phenomenons and human behaviours in the long history, with data-driven machine learning models, has emerged as an effective way to mitigate the shortage of training data, to increase models' generalizability and to ensure the physical plausibility of results. In this paper, we survey an abundant number of recent works in PIML and summarize them from three aspects: (1) motivations of PIML, (2) physics knowledge in PIML, (3) methods of physics knowl…
Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Fraunhofer IAIS, Institute for Intelligent Analysis and Information Systems（弗劳恩霍夫智能分析与信息系统研究所）；Fraunhofer SCAI, Institute for Algorithms and Scientific Computing（弗劳恩霍夫算法与科学计算研究所）；Fraunhofer ITWM, Institute for Industrial Mathematics（弗劳恩霍夫工业数学研究所）；Fraunhofer IOSB, Institute for Optronics, System Technologies and Image Exploitation（弗劳恩霍夫光电子学、系统技术与图像开发研究所）
Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for informed machine learning which illustrates its building blocks and distinguishes it from conventional machine learning. We introduce a taxonomy that serves as a classification framework for informed mac…
ETH Zürich（苏黎世联邦理工学院）；ETH AI Center（苏黎世联邦理工学院人工智能中心）
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. These bounds guarantee that PINNs and (physics-informed) DeepONets or FNOs will efficiently approximate the underlying solution or solution operator of generic partial differential equations (PDEs). Our framework utilizes existing neural network approximation results to obtain bounds on more involved learning architectures for PDEs. We illustrate …
We introduce Physics Informed Symbolic Networks (PISN) which utilize physics-informed loss to obtain a symbolic solution for a system of Partial Differential Equations (PDE). Given a context-free grammar to describe the language of symbolic expressions, we propose to use weighted sum as continuous approximation for selection of a production rule. We use this approximation to define multilayer symbolic networks. We consider Kovasznay flow (Navier-Stokes) and two-dimensional viscous Burger's equations to illustrate that PISN are able to provide a performance comparable to PINNs across various st…
The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The $L^2$ Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With…
University of Twente（特文特大学）；Massachusetts Institute of Technology（麻省理工学院）
An energy-based a posteriori error bound is proposed for the physics-informed neural network solutions of elasticity problems. An admissible displacement-stress solution pair is obtained from a mixed form of physics-informed neural networks, and the proposed error bound is formulated as the constitutive relation error defined by the solution pair. Such an error estimator provides an upper bound of the global error of neural network discretization. The bounding property, as well as the asymptotic behavior of the physics-informed neural network solutions, are studied in a demonstrating example.
Adaptive Physics-informed Neural Networks: A Survey
Institute for Artificial Intelligence, University of Stuttgart（斯图加特大学人工智能研究所）；Robert Bosch GmbH（罗伯特·博世有限公司）；IMPRS-IS（国际研究生学院智能系统（IMPRS-IS））；NEC Labs Europe（NEC欧洲实验室）；SimTech（仿真技术研究所（SimTech））
We prove rigorous bounds on the errors resulting from the approximation of the incompressible Navier-Stokes equations with (extended) physics informed neural networks. We show that the underlying PDE residual can be made arbitrarily small for tanh neural networks with two hidden layers. Moreover, the total error can be estimated in terms of the training error, network size and number of quadrature points. The theory is illustrated with numerical experiments.
University of Nevada, Las Vegas（内华达大学拉斯维加斯分校）；University of Arkansas（阿肯色大学）；Lehigh University（里海大学）
In this paper, we show a physics-informed neural network solver for the time-dependent surface PDEs. Unlike the traditional numerical solver, no extension of PDE and mesh on the surface is needed. We show a simplified prior estimate of the surface differential operators so that PINN's loss value will be an indicator of the residue of the surface PDEs. Numerical experiments verify efficacy of our algorithm.
Technical University of Denmark（丹麦技术大学）；The Danish Hydrocarbon Research and Technology Centre（丹麦碳氢化合物研究与技术中心）
This paper presents the potential of applying physics-informed neural networks for solving nonlinear multiphysics problems, which are essential to many fields such as biomedical engineering, earthquake prediction, and underground energy harvesting. Specifically, we investigate how to extend the methodology of physics-informed neural networks to solve both the forward and inverse problems in relation to the nonlinear diffusivity and Biot's equations. We explore the accuracy of the physics-informed neural networks with different training example sizes and choices of hyperparameters. The impacts …
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
Xi'an University of Architecture and Technology（西安建筑科技大学）；Xidian University（西安电子科技大学）
Big-data-based artificial intelligence (AI) supports profound evolution in almost all of science and technology. However, modeling and forecasting multi-physical systems remain a challenge due to unavoidable data scarcity and noise. Improving the generalization ability of neural networks by "teaching" domain knowledge and developing a new generation of models combined with the physical laws have become promising areas of machine learning research. Different from "deep" fully-connected neural networks embedded with physical information (PINN), a novel shallow framework named physics-informed co…
Whilst the partial differential equations that govern the dynamics of our world have been studied in great depth for centuries, solving them for complex, high-dimensional conditions and domains still presents an incredibly large mathematical and computational challenge. Analytical methods can be cumbersome to utilise, and numerical methods can lead to errors and inaccuracies. On top of this, sometimes we lack the information or knowledge to pose the problem well enough to apply these kinds of methods. Here, we present a new approach to approximating the solution to physical systems - physics-i…
Partition of Unity Physics-Informed Neural Networks (POU-PINNs): An Unsupervised Framework for Physics-Informed Domain Decomposition and Mixtures of Experts
The University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；Clarkson University（克拉克森大学）；Texas A&M University in Kingsville（德克萨斯农工大学金斯维尔分校）
Integrating physical inductive biases into machine learning can improve model generalizability. We generalize the successful paradigm of physics-informed learning (PIL) into a more general framework that also includes what we term physics-augmented learning (PAL). PIL and PAL complement each other by handling discriminative and generative properties, respectively. In numerical experiments, we show that PAL performs well on examples where PIL is inapplicable or inefficient.
PIDT: Physics-Informed Digital Twin for Optical Fiber Parameter Estimation
Dept. of Electrical Engineering, Chalmers Univ. of Technology, Sweden（电气工程系，瑞典查尔姆斯理工大学）；Dept. of Microtechnology and Nanoscience, Chalmers Univ. of Technology, Sweden（微电子与纳米科技系，瑞典查尔姆斯理工大学）
Conformal mapping based Physics-informed neural networks for designing neutral inclusions
Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology（韩国科学技术院数学科学系）；Department of Artificial Intelligence, Chung-Ang University（Chung-Ang 大学人工智能系）
Physics-informed Diffusion Generation for Geomagnetic Map Interpolation
State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；High-Performance Intelligent Computing Research Center for Ultra-Large Scale Graph Data（超大规模图数据高性能智能计算研究中心）；School of Computer and Computing Science（计算机与计算科学学院）；Hangzhou City University（杭州市大学）；Zhejiang Lab（浙江实验室）
Unsupervised Physics-Informed Operator Learning through Multi-Stage Curriculum Training
organization= Department of Civil；organization= QC Ware Corp. , city= Paris , country= France；organization= Department of Computer Science, KTH Royal Institute of Technology , city= Stockholm , country= Sweden；organization= IRIF, CNRS \& Université Paris Cité , city= Paris , country= France
University of Maryland（马里兰大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Considerable research has been devoted to deep learning-based predictive models for system prognostics and health management in the reliability and safety community. However, there is limited study on the utilization of deep learning for system reliability assessment. This paper aims to bridge this gap and explore this new interface between deep learning and system reliability assessment by exploiting the recent advances of physics-informed deep learning. Particularly, we present an approach to frame system reliability assessment in the context of physics-informed deep learning and discuss the…
University of Cambridge（剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）；Imperial College London（帝国理工学院）；Technical University of Munich（慕尼黑工业大学）
We propose the Automatic-differentiated Physics-Informed Echo State Network (API-ESN). The network is constrained by the physical equations through the reservoir's exact time-derivative, which is computed by automatic differentiation. As compared to the original Physics-Informed Echo State Network, the accuracy of the time-derivative is increased by up to seven orders of magnitude. This increased accuracy is key in chaotic dynamical systems, where errors grows exponentially in time. The network is showcased in the reconstruction of unmeasured (hidden) states of a chaotic system. The API-ESN el…
This paper presents NSGA-PINN, a multi-objective optimization framework for effective training of Physics-Informed Neural Networks (PINNs). The proposed framework uses the Non-dominated Sorting Genetic Algorithm (NSGA-II) to enable traditional stochastic gradient optimization algorithms (e.g., ADAM) to escape local minima effectively. Additionally, the NSGA-II algorithm enables satisfying the initial and boundary conditions encoded into the loss function during physics-informed training precisely. We demonstrate the effectiveness of our framework by applying NSGA-PINN to several ordinary and p…
Massachusetts Institute of Technology（麻省理工学院）；Sandia National Laboratories（桑迪亚国家实验室）
Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networ…
Advanced Research Center, National Fusion Research Institute (NFRI)（国家聚变研究所高级研究中心）
Inspired by recent theoretical arguments, physics-informed echo state network (ESN) is discussed on the attempt to train a reservoir model absolutely in physics-informed manner. As the plainest work on such a purpose, an ODE (ordinary differential equation) approximator is designed to replicate the solution in sequence with respect to the recurrent evaluations. On the principal invariance of differential equations, the constraint in recurrence just takes shape to secure a proper regression method for the ESN-based ODE approximator. After then, the actual training process is established on the …
