Applications of physics-informed scientific machine learning in subsurface science: A survey
Bureau of Economic Geology, Jackson School of Geosciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校杰克逊地球科学学院经济地质局）；Sandia National Laboratories（桑迪亚国家实验室）；Leidos（Leidos公司）；School of Earth Resources, China University of Geosciences（中国地质大学地球资源学院）
Alexander Y. Sun, Hongkyu Yoon, Chung-Yan Shih, Zhi Zhong
Geosystems are geological formations altered by humans activities such as fossil energy exploration, waste disposal, geologic carbon sequestration, and renewable energy generation. Geosystems also represent a critical link in the global water-energy nexus, providing both the source and buffering mechanisms for enabling societal adaptation to climate variability and change. The responsible use and exploration of geosystems are thus critical to the geosystem governance, which in turn depends on the efficient monitoring, risk assessment, and decision support tools for practical implementation. Fast advances in machine learning (ML) algorithms and novel sensing technologies in recent years have presented new opportunities for the subsurface research community to improve the efficacy and transparency of geosystem governance. Although recent studies have shown the great promise of scientific ML (SciML) models, questions remain on how to best leverage ML in the management of geosystems, which are typified by multiscality, high-dimensionality, and data resolution inhomogeneity. This survey will provide a systematic review of the recent development and applications of domain-aware SciML in geosystem researches, with an emphasis on how the accuracy, interpretability, scalability, defensibility, and generalization skill of ML approaches can be improved to better serve the geoscientific community.
Pacific Northwest National Laboratory（太平洋西北国家实验室）；INTERA Incorporated（INTERA公司）
Data assimilation for parameter and state estimation in subsurface transport problems remains a significant challenge due to the sparsity of measurements, the heterogeneity of porous media, and the high computational cost of forward numerical models. We present a physics-informed deep neural networks (DNNs) machine learning method for estimating space-dependent hydraulic conductivity, hydraulic head, and concentration fields from sparse measurements. In this approach, we employ individual DNNs to approximate the unknown parameters (e.g., hydraulic conductivity) and states (e.g., hydraulic head…
Physics-informed waveform inversion using pretrained wavefield neural operators
Physical Science and Engineering division, King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学物理科学与工程部门）；School of Engineering and Applied Science, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）
In this paper, we introduce SciANN, a Python package for scientific computing and physics-informed deep learning using artificial neural networks. SciANN uses the widely used deep-learning packages Tensorflow and Keras to build deep neural networks and optimization models, thus inheriting many of Keras's functionalities, such as batch optimization and model reuse for transfer learning. SciANN is designed to abstract neural network construction for scientific computations and solution and discovery of partial differential equations (PDE) using the physics-informed neural networks (PINN) archite…
Texas State University（德克萨斯州立大学）；University of Minho（米尼奥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Texas A&M University（德克萨斯农工大学）
Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, including fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints d…
Physics-Informed Neural Networks for Predicting Hydrogen Sorption in Geological Formations: Thermodynamically Constrained Deep Learning Integrating Classical Adsorption Theory
University of Oslo（奥斯陆大学）；SINTEF Industry（挪威科技工业研究院）；China University of Petroleum, Beijing（中国石油大学（北京））
University of Naples Federico II（那不勒斯费德里科二世大学）；SISSA – International School for Advanced Studies（国际高等研究学院（SISSA））；University of Colorado Boulder（科罗拉多大学博尔德分校）
Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related adv…
RAMS: Residual-based adversarial-gradient moving sample method for scientific machine learning in solving partial differential equations
Department of Civil and Environmental Engineering, Hong Kong Polytechnic University（香港理工大学土木与环境工程系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）
Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning
Elmore Family School of Electrical and Computer Engineering（埃洛姆家族电气与计算机工程学院）；Purdue University（普渡大学）；Department of Mathematics（数学系）；Department of Mathematics & School of Mechanical Engineering（数学系与机械工程学院）
EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Adaptive Physics-informed Neural Networks: A Survey
Institute for Artificial Intelligence, University of Stuttgart（斯图加特大学人工智能研究所）；Robert Bosch GmbH（罗伯特·博世有限公司）；IMPRS-IS（国际研究生学院智能系统（IMPRS-IS））；NEC Labs Europe（NEC欧洲实验室）；SimTech（仿真技术研究所（SimTech））
The widespread use of neural networks across different scientific domains often involves constraining them to satisfy certain symmetries, conservation laws, or other domain knowledge. Such constraints are often imposed as soft penalties during model training and effectively act as domain-specific regularizers of the empirical risk loss. Physics-informed neural networks is an example of this philosophy in which the outputs of deep neural networks are constrained to approximately satisfy a given set of partial differential equations. In this work we review recent advances in scientific machine l…
Learn and Verify: A Framework for Rigorous Verification of Physics-Informed Neural Networks
Global Center for Science and Engineering（全球科学与工程中心）；Waseda University（早稻田大学）；Faculty of Engineering（工学院）；Tokyo University of Agriculture and Technology（东京农业大学）
Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of North Carolina, Charlotte（北卡罗来纳大学夏洛特分校）；Delft University of Technology（代尔夫特理工大学）；University of Washington, Applied Mathematics（华盛顿大学应用数学系）；Brown University, Applied Mathematics（布朗大学应用数学系）
Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能系，弗劳恩霍夫海因里希·赫兹研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD -- Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）
Defense Innovation Institute, Chinese Academy of Military Science（军事科学院国防创新研究院）
Physics Informed Neural Network (PINN) is a scientific computing framework used to solve both forward and inverse problems modeled by Partial Differential Equations (PDEs). This paper introduces IDRLnet, a Python toolbox for modeling and solving problems through PINN systematically. IDRLnet constructs the framework for a wide range of PINN algorithms and applications. It provides a structured way to incorporate geometric objects, data sources, artificial neural networks, loss metrics, and optimizers within Python. Furthermore, it provides functionality to solve noisy inverse problems, variatio…
Revisiting Heat Flux Analysis of Tungsten Monoblock Divertor on EAST using Physics-Informed Neural Network
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Tongling University（铜陵大学）；Institute of Plasma Physics, Chinese Academy of Sciences（中国科学院等离子体物理研究所）
The advent of scientific machine learning (SciML) has opened up a new field with many promises and challenges in the field of simulation science by developing approaches at the interface of physics- and data-based modelling. To this end, physics-informed neural networks (PINNs) have been introduced in recent years, which cope for the scarcity in training data by incorporating physics knowledge of the problem at so-called collocation points. In this work, we investigate the prediction performance of PINNs with respect to the number of collocation points used to enforce the physics-based penalty…
NTT Human Informatics Labs（NTT人类信息学实验室）；NTT Communication Science Labs（NTT通信科学实验室）
Opinion formation and propagation are crucial phenomena in social networks and have been extensively studied across several disciplines. Traditionally, theoretical models of opinion dynamics have been proposed to describe the interactions between individuals (i.e., social interaction) and their impact on the evolution of collective opinions. Although these models can incorporate sociological and psychological knowledge on the mechanisms of social interaction, they demand extensive calibration with real data to make reliable predictions, requiring much time and effort. Recently, the widespread …
University of Oxford（牛津大学）；Harvard University（哈佛大学）
Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solving differential equations, transfer learning has been under explored. In this study, we present a general framework for transfer learning PINNs that results in one-shot inference for linear systems of …
Gryphon Center for Artificial Intelligence and Theoretical Sciences（格里芬人工智能与理论科学中心）；Georgia Institute of Technology（佐治亚理工学院）；University of Pennsylvania（宾夕法尼亚大学）
We present a novel class of Physics-Informed Neural Networks that is formulated based on the principles of Evidential Deep Learning, where the model incorporates uncertainty quantification by learning parameters of a higher-order distribution. The dependent and trainable variables of the PDE residual loss and data-fitting loss terms are recast as functions of the hyperparameters of an evidential prior distribution. Our model is equipped with an information-theoretic regularizer that contains the Kullback-Leibler divergence between two inverse-gamma distributions characterizing predictive uncer…
Nanyang Technological University（南洋理工大学）；Institute of Bioengineering and Bioimaging, A*STAR（新加坡科技研究局生物工程与生物成像研究所）；Bioinformatics Institute, A*STAR（新加坡科技研究局生物信息学研究所）；Institute of High Performance Computing（高性能计算研究所）；Centre for Frontier AI Research（前沿人工智能研究中心）
Reaction-diffusion (Turing) systems are fundamental to the formation of spatial patterns in nature and engineering. These systems are governed by a set of non-linear partial differential equations containing parameters that determine the rate of constituent diffusion and reaction. Critically, these parameters, such as diffusion coefficient, heavily influence the mode and type of the final pattern, and quantitative characterization and knowledge of these parameters can aid in bio-mimetic design or understanding of real-world systems. However, the use of numerical methods to infer these paramete…
Peng Cheng Laboratory（鹏城实验室）；The University of Tokyo（东京大学）；Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）；Malardalen University（马拉达伦大学）；LocationMind Inc.（LocationMind公司）
Machine learning models have been successfully used in many scientific and engineering fields. However, it remains difficult for a model to simultaneously utilize domain knowledge and experimental observation data. The application of knowledge-based symbolic AI represented by an expert system is limited by the expressive ability of the model, and data-driven connectionism AI represented by neural networks is prone to produce predictions that violate physical mechanisms. In order to fully integrate domain knowledge with observations, and make full use of the prior information and the strong fit…
Solving Euler equations with Multiple Discontinuities via Separation-Transfer Physics-Informed Neural Networks
Graduate School, China Academy of Engineering Physics（中国工程物理研究院研究生院）；China Academy of Engineering Physics（中国工程物理研究院）；Zhejiang Institute of Modern Physics（浙江现代物理研究所）；School of Physics, Zhejiang University（浙江大学物理学院）；Beijing Computational Science Research Center（北京计算科学研究院）
Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientific problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but a…
