Scientific foundation models (SciFMs) aim to learn generalizable representations of physical systems governed by partial differential equations (PDEs), enabling transfer across tasks and domains. While physics-informed methods, which leverage PDE residuals as supervisory signals, have shown promise in scientific machine learning (SciML) for improving accuracy and reducing data requirements, their potential in the context of SciFMs remains relatively unexplored.
In this evaluation study, we investigate whether (and how) physics-informed pre-training improves the generalization, robustness, and data efficiency of SciFMs. We conduct systematic experiments across a diverse set of PDEs, ranging from simple problems with periodic boundary conditions to more challenging systems such as the Navier-Stokes equations and non-periodic geometries.
Our results show that physics-informed pre-training provides clear benefits in ``nice,'' e.g., structured, well-aligned settings: it enhances generalization and reduces data dependence, compared to data-only pre-training. However, these advantages diminish significantly as the downstream tasks become ``harder,'' e.g., as they involve discontinuities or deviate from the pre-training distribution. In complex or structurally different problems, such as those involving new boundary conditions or PDE operators, physics-informed models may perform only on par with---or even worse---than data-driven baselines.
While residual-based pre-training helps in idealized regimes, realizing broadly transferable SciFMs will likely require subtler spatiotemporal inductive biases and more principled integration of physical knowledge into model architectures.
Texas State University（德克萨斯州立大学）；University of Minho（米尼奥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Texas A&M University（德克萨斯农工大学）
Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, including fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints d…
In this paper, we introduce SciANN, a Python package for scientific computing and physics-informed deep learning using artificial neural networks. SciANN uses the widely used deep-learning packages Tensorflow and Keras to build deep neural networks and optimization models, thus inheriting many of Keras's functionalities, such as batch optimization and model reuse for transfer learning. SciANN is designed to abstract neural network construction for scientific computations and solution and discovery of partial differential equations (PDE) using the physics-informed neural networks (PINN) archite…
Bureau of Economic Geology, Jackson School of Geosciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校杰克逊地球科学学院经济地质局）；Sandia National Laboratories（桑迪亚国家实验室）；Leidos（Leidos公司）；School of Earth Resources, China University of Geosciences（中国地质大学地球资源学院）
Geosystems are geological formations altered by humans activities such as fossil energy exploration, waste disposal, geologic carbon sequestration, and renewable energy generation. Geosystems also represent a critical link in the global water-energy nexus, providing both the source and buffering mechanisms for enabling societal adaptation to climate variability and change. The responsible use and exploration of geosystems are thus critical to the geosystem governance, which in turn depends on the efficient monitoring, risk assessment, and decision support tools for practical implementation. Fa…
Northwest Institute of Nuclear Technology（西北核技术研究院）；Northwestern Polytechnical University（西北工业大学）；Xidian University（西安电子科技大学）；Central China Normal University（华中师范大学）；Shaanxi Normal University（陕西师范大学）；Institute of Science Tokyo（东京科学大学）
本文发现神经网络对微小扰动高度敏感，导致输出显著偏差，并在天气预报、流体动力学等五个领域验证了该脆弱性是 AI 系统的普遍特征，揭示了科学计算中的潜在风险。
University of Naples Federico II（那不勒斯费德里科二世大学）；SISSA – International School for Advanced Studies（国际高等研究学院（SISSA））；University of Colorado Boulder（科罗拉多大学博尔德分校）
Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related adv…
Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning
Elmore Family School of Electrical and Computer Engineering（埃洛姆家族电气与计算机工程学院）；Purdue University（普渡大学）；Department of Mathematics（数学系）；Department of Mathematics & School of Mechanical Engineering（数学系与机械工程学院）
RAMS: Residual-based adversarial-gradient moving sample method for scientific machine learning in solving partial differential equations
Department of Civil and Environmental Engineering, Hong Kong Polytechnic University（香港理工大学土木与环境工程系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）
EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学先进交叉科学学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Adaptive Physics-informed Neural Networks: A Survey
Institute for Artificial Intelligence, University of Stuttgart（斯图加特大学人工智能研究所）；Robert Bosch GmbH（罗伯特·博世有限公司）；IMPRS-IS（国际研究生学院智能系统（IMPRS-IS））；NEC Labs Europe（NEC欧洲实验室）；SimTech（仿真技术研究所（SimTech））
Improved physics-informed neural network in mitigating gradient related failures
College of Applied Mathematics, Chengdu University of Information Technology（成都信息工程大学应用数学学院）；College of Mathematics, Sichuan University（四川大学数学学院）；School of Sciences, Southwest Petroleum University（西南石油大学理学院）
Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of North Carolina, Charlotte（北卡罗来纳大学夏洛特分校）；Delft University of Technology（代尔夫特理工大学）；University of Washington, Applied Mathematics（华盛顿大学应用数学系）；Brown University, Applied Mathematics（布朗大学应用数学系）
National Innovation Institute of Defense Technology, Chinese Academy of Military Science（中国人民解放军军事科学院国防科技创新研究院）
Physics-informed neural networks (PINNs) have been widely used to solve various scientific computing problems. However, large training costs limit PINNs for some real-time applications. Although some works have been proposed to improve the training efficiency of PINNs, few consider the influence of initialization. To this end, we propose a New Reptile initialization based Physics-Informed Neural Network (NRPINN). The original Reptile algorithm is a meta-learning initialization method based on labeled data. PINNs can be trained with less labeled data or even without any labeled data by adding p…
Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?
Department of Radiophysics, University of Nizhny Novgorod（尼日尼诺夫戈罗德大学无线电物理系）；Huawei Nizhny Novgorod Research Center（华为尼日尼诺夫戈罗德研究中心）；Skolkovo Institute of Science and Technology（斯克尔科沃科学技术研究院）
Learn and Verify: A Framework for Rigorous Verification of Physics-Informed Neural Networks
Global Center for Science and Engineering（全球科学与工程中心）；Waseda University（早稻田大学）；Faculty of Engineering（工学院）；Tokyo University of Agriculture and Technology（东京农业大学）
Revisiting Heat Flux Analysis of Tungsten Monoblock Divertor on EAST using Physics-Informed Neural Network
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Tongling University（铜陵大学）；Institute of Plasma Physics, Chinese Academy of Sciences（中国科学院等离子体物理研究所）
The widespread use of neural networks across different scientific domains often involves constraining them to satisfy certain symmetries, conservation laws, or other domain knowledge. Such constraints are often imposed as soft penalties during model training and effectively act as domain-specific regularizers of the empirical risk loss. Physics-informed neural networks is an example of this philosophy in which the outputs of deep neural networks are constrained to approximately satisfy a given set of partial differential equations. In this work we review recent advances in scientific machine l…
Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能系，弗劳恩霍夫海因里希·赫兹研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD -- Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）
Defense Innovation Institute, Chinese Academy of Military Science（军事科学院国防创新研究院）
Physics Informed Neural Network (PINN) is a scientific computing framework used to solve both forward and inverse problems modeled by Partial Differential Equations (PDEs). This paper introduces IDRLnet, a Python toolbox for modeling and solving problems through PINN systematically. IDRLnet constructs the framework for a wide range of PINN algorithms and applications. It provides a structured way to incorporate geometric objects, data sources, artificial neural networks, loss metrics, and optimizers within Python. Furthermore, it provides functionality to solve noisy inverse problems, variatio…
The advent of scientific machine learning (SciML) has opened up a new field with many promises and challenges in the field of simulation science by developing approaches at the interface of physics- and data-based modelling. To this end, physics-informed neural networks (PINNs) have been introduced in recent years, which cope for the scarcity in training data by incorporating physics knowledge of the problem at so-called collocation points. In this work, we investigate the prediction performance of PINNs with respect to the number of collocation points used to enforce the physics-based penalty…
Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs). However, they often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under stochastic gradient descent with momentum (SGDM). This demonstrates SGDM significantly reduces the effect of spectral bias. We have also examined why …
University of Oxford（牛津大学）；Harvard University（哈佛大学）
Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solving differential equations, transfer learning has been under explored. In this study, we present a general framework for transfer learning PINNs that results in one-shot inference for linear systems of …
NTT Human Informatics Labs（NTT人类信息学实验室）；NTT Communication Science Labs（NTT通信科学实验室）
Opinion formation and propagation are crucial phenomena in social networks and have been extensively studied across several disciplines. Traditionally, theoretical models of opinion dynamics have been proposed to describe the interactions between individuals (i.e., social interaction) and their impact on the evolution of collective opinions. Although these models can incorporate sociological and psychological knowledge on the mechanisms of social interaction, they demand extensive calibration with real data to make reliable predictions, requiring much time and effort. Recently, the widespread …
Solving Euler equations with Multiple Discontinuities via Separation-Transfer Physics-Informed Neural Networks
Graduate School, China Academy of Engineering Physics（中国工程物理研究院研究生院）；China Academy of Engineering Physics（中国工程物理研究院）；Zhejiang Institute of Modern Physics（浙江现代物理研究所）；School of Physics, Zhejiang University（浙江大学物理学院）；Beijing Computational Science Research Center（北京计算科学研究院）
Gryphon Center for Artificial Intelligence and Theoretical Sciences（格里芬人工智能与理论科学中心）；Georgia Institute of Technology（佐治亚理工学院）；University of Pennsylvania（宾夕法尼亚大学）
We present a novel class of Physics-Informed Neural Networks that is formulated based on the principles of Evidential Deep Learning, where the model incorporates uncertainty quantification by learning parameters of a higher-order distribution. The dependent and trainable variables of the PDE residual loss and data-fitting loss terms are recast as functions of the hyperparameters of an evidential prior distribution. Our model is equipped with an information-theoretic regularizer that contains the Kullback-Leibler divergence between two inverse-gamma distributions characterizing predictive uncer…
Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientific problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but a…
Nanyang Technological University（南洋理工大学）；Institute of Bioengineering and Bioimaging, A*STAR（新加坡科技研究局生物工程与生物成像研究所）；Bioinformatics Institute, A*STAR（新加坡科技研究局生物信息学研究所）；Institute of High Performance Computing（高性能计算研究所）；Centre for Frontier AI Research（前沿人工智能研究中心）
Reaction-diffusion (Turing) systems are fundamental to the formation of spatial patterns in nature and engineering. These systems are governed by a set of non-linear partial differential equations containing parameters that determine the rate of constituent diffusion and reaction. Critically, these parameters, such as diffusion coefficient, heavily influence the mode and type of the final pattern, and quantitative characterization and knowledge of these parameters can aid in bio-mimetic design or understanding of real-world systems. However, the use of numerical methods to infer these paramete…
Unsupervised Physics-Informed Operator Learning through Multi-Stage Curriculum Training
organization= Department of Civil；organization= QC Ware Corp. , city= Paris , country= France；organization= Department of Computer Science, KTH Royal Institute of Technology , city= Stockholm , country= Sweden；organization= IRIF, CNRS \& Université Paris Cité , city= Paris , country= France
Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators
Brown University（布朗大学）；Huazhong University of Science and Technology（华中科技大学）；Institute of Interdisciplinary Research for Mathematics and Applied Science, School of Mathematics and Statistics, Huazhong University of Science and Technology（华中科技大学数学与统计学院数学与应用科学交叉研究院）
Virginia Tech（弗吉尼亚理工大学）；University of Michigan（密歇根大学）
Physics-informed Neural Networks (PINNs) are gaining attention in the engineering and scientific literature for solving a range of differential equations with applications in weather modeling, healthcare, manufacturing, etc. Poor scalability is one of the barriers to utilizing PINNs for many real-world problems. To address this, a Self-scalable tanh (Stan) activation function is proposed for the PINNs. The proposed Stan function is smooth, non-saturating, and has a trainable parameter. During training, it can allow easy flow of gradients to compute the required derivatives and also enable syst…
