Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
Peng Cheng Laboratory（鹏城实验室）；The University of Tokyo（东京大学）；Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）；Malardalen University（马拉达伦大学）；LocationMind Inc.（LocationMind公司）
Machine learning models have been successfully used in many scientific and engineering fields. However, it remains difficult for a model to simultaneously utilize domain knowledge and experimental observation data. The application of knowledge-based symbolic AI represented by an expert system is limited by the expressive ability of the model, and data-driven connectionism AI represented by neural networks is prone to produce predictions that violate physical mechanisms. In order to fully integrate domain knowledge with observations, and make full use of the prior information and the strong fitting ability of neural networks, this study proposes theory-guided hard constraint projection (HCP). This model converts physical constraints, such as governing equations, into a form that is easy to handle through discretization, and then implements hard constraint optimization through projection. Based on rigorous mathematical proofs, theory-guided HCP can ensure that model predictions strictly conform to physical mechanisms in the constraint patch. The performance of the theory-guided HCP is verified by experiments based on the heterogeneous subsurface flow problem. Due to the application of hard constraints, compared with fully connected neural networks and soft constraint models, such as theory-guided neural networks and physics-informed neural networks, theory-guided HCP requires fewer data, and achieves higher prediction accuracy and stronger robustness to noisy observations.
University of Minnesota（明尼苏达大学）；University of Cincinnati（辛辛那提大学）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Northeastern University（东北大学）；North Carolina State University（北卡罗来纳州立大学）
Data science models, although successful in a number of commercial domains, have had limited applicability in scientific problems involving complex physical phenomena. Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of data science models in enabling scientific discovery. The overarching vision of TGDS is to introduce scientific consistency as an essential component for learning generalizable models. Further, by producing scientifically interpretable models, TGDS aims to advance our scientific un…
Texas State University（德克萨斯州立大学）；University of Minho（米尼奥大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Texas A&M University（德克萨斯农工大学）
Recent breakthroughs in computing power have made it feasible to use machine learning and deep learning to advance scientific computing in many fields, including fluid mechanics, solid mechanics, materials science, etc. Neural networks, in particular, play a central role in this hybridization. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data is sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints d…
BayesEvolve: Explicit Belief States for Autonomous Scientific Discovery
Pfizer（辉瑞公司）；Independent Researcher（独立研究员）；University of California San Diego（加州大学圣地亚哥分校）；Institute of Humanities and Social Science Data, Fudan University（复旦大学人文社会科学数据研究院）
Budget-Sensitive Discovery Scoring: A Formally Verified Framework for Evaluating AI-Guided Scientific Selection
Indian Institute of Information Technology Allahabad (IIITA)（印度阿瓦德理工学院信息学院（IIITA））；National Institute of Electronics & Information Technology (NIELIT)（国家电子与信息科技研究所（NIELIT））
Oklahoma State University（俄克拉荷马州立大学）；Texas A&M University-San Antonio（德克萨斯农工大学圣安东尼奥分校）；Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（SINTEF数字研究所）
Recent applications of machine learning, in particular deep learning, motivate the need to address the generalizability of the statistical inference approaches in physical sciences. In this letter, we introduce a modular physics guided machine learning framework to improve the accuracy of such data-driven predictive engines. The chief idea in our approach is to augment the knowledge of the simplified theories with the underlying learning process. To emphasise on their physical importance, our architecture consists of adding certain features at intermediate layers rather than in the input layer…
Deploy-Master: Automating the Deployment of 50,000+ Agent-Ready Scientific Tools in One Day
DP Technology Beijing China（DP技术北京中国）；AI for Science Institute Beijing China（AI for Science研究院北京中国）；Shanghai Jiao Tong University Shanghai China（上海交通大学上海中国）
Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
This study presents a broad perspective of hybrid process modeling and optimization combining the scientific knowledge and data analytics in bioprocessing and chemical engineering with a science-guided machine learning (SGML) approach. We divide the approach into two major categories. The first refers to the case where a data-based ML model compliments and makes the first-principle science-based model more accurate in prediction, and the second corresponds to the case where scientific knowledge helps make the ML model more scientifically consistent. We present a detailed review of scientific a…
Virginia Tech（弗吉尼亚理工学院暨州立大学）；U.S. Geological Survey（美国地质调查局）；University of Minnesota（明尼苏达大学）
This paper introduces a framework for combining scientific knowledge of physics-based models with neural networks to advance scientific discovery. This framework, termed physics-guided neural networks (PGNN), leverages the output of physics-based model simulations along with observational features in a hybrid modeling setup to generate predictions using a neural network architecture. Further, this framework uses physics-based loss functions in the learning objective of neural networks to ensure that the model predictions not only show lower errors on the training set but are also scientificall…
University of Minnesota（明尼苏达大学）；United States Geologic Survey（美国地质调查局）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
In this paper, we introduce a novel framework for combining scientific knowledge within physics-based models and recurrent neural networks to advance scientific discovery in many dynamical systems. We will first describe the use of outputs from physics-based models in learning a hybrid-physics-data model. Then, we further incorporate physical knowledge in real-world dynamical systems as additional constraints for training recurrent neural networks. We will apply this approach on modeling lake temperature and quality where we take into account the physical constraints along both the depth dimen…
University of Minnesota, Twin Cities（明尼苏达大学双城分校）；Virginia Tech（弗吉尼亚理工大学）；U.S. Geological Survey（美国地质调查局）；University of Minnesota（明尼苏达大学）
Physics-based models of dynamical systems are often used to study engineering and environmental systems. Despite their extensive use, these models have several well-known limitations due to simplified representations of the physical processes being modeled or challenges in selecting appropriate parameters. While-state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and phys…
Physics-Guided Multimodal Transformers are the Necessary Foundation for the Next Generation of Meteorological Science
School of Artificial Intelligence, Beijing University of Posts；Huawei Noah's Ark Lab；Department of Earth System Science, Tsinghua University；College of Computing \& Data Science, Nanyang Technological University；State Key Laboratory of Networking；Switching Technology, Beijing University of Posts
Learning-guided Kansa collocation for forward and inverse PDEs beyond linearity
Department of Computer Science and Technology（计算机科学与技术系）；Department of Applied Mathematics and Theoretical Physics（应用数学与理论物理系）；University of Cambridge（剑桥大学）
The measured spatiotemporal response of various physical processes is utilized to infer the governing partial differential equations (PDEs). We propose SimultaNeous Basis Function Approximation and Parameter Estimation (SNAPE), a technique of parameter estimation of PDEs that is robust against high levels of noise nearly 100 %, by simultaneously fitting basis functions to the measured response and estimating the parameters of both ordinary and partial differential equations. The domain knowledge of the general multidimensional process is used as a constraint in the formulation of the optimizat…
Pacific Northwest National Laboratory（太平洋西北国家实验室）；Washington State University – Tri-Cities（华盛顿州立大学三城分校）
Artificial intelligence (AI) promises to reshape scientific inquiry and enable breakthrough discoveries in areas such as energy storage, quantum computing, and biomedicine. Scanning transmission electron microscopy (STEM), a cornerstone of the study of chemical and materials systems, stands to benefit greatly from AI-driven automation. However, present barriers to low-level instrument control, as well as generalizable and interpretable feature detection, make truly automated microscopy impractical. Here, we discuss the design of a closed-loop instrument control platform guided by emerging spar…
The Ohio State University（俄亥俄州立大学）；Binghamton University（宾汉姆顿大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Virginia Tech（弗吉尼亚理工大学）
Given their ability to effectively learn non-linear mappings and perform fast inference, deep neural networks (NNs) have been proposed as a viable alternative to traditional simulation-driven approaches for solving high-dimensional eigenvalue equations (HDEs), which are the foundation for many scientific applications. Unfortunately, for the learned models in these scientific applications to achieve generalization, a large, diverse, and preferably annotated dataset is typically needed and is computationally expensive to obtain. Furthermore, the learned models tend to be memory- and compute-inte…
University of Minnesota（明尼苏达大学）；University of Pittsburgh（匹兹堡大学）
There is a growing consensus that solutions to complex science and engineering problems require novel methodologies that are able to integrate traditional physics-based modeling approaches with state-of-the-art machine learning (ML) techniques. This paper provides a structured overview of such techniques. Application-centric objective areas for which these approaches have been applied are summarized, and then classes of methodologies used to construct physics-guided ML models and hybrid physics-ML frameworks are described. We then provide a taxonomy of these existing techniques, which uncovers…
Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
Carnegie Mellon University（卡内基梅隆大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；Hefei University of Technology（合肥工业大学）；RealAI（瑞莱智慧）；Tsinghua University（清华大学）
Aragón Institute of Engineering Research (I3A)（阿拉贡工程研究所（I3A））；University of Zaragoza（萨拉戈萨大学）；Escuela de Ingeniería y Arquitectura (EINA)（工程与建筑学院（EINA））
Substitution of well-grounded theoretical models by data-driven predictions is not as simple in engineering and sciences as it is in social and economic fields. Scientific problems suffer most times from paucity of data, while they may involve a large number of variables and parameters that interact in complex and non-stationary ways, obeying certain physical laws. Moreover, a physically-based model is not only useful for making predictions, but to gain knowledge by the interpretation of its structure, parameters, and mathematical properties. The solution to these shortcomings seems to be the …
We introduce a multi-agent meta-modeling game to generate data, knowledge, and models that make predictions on constitutive responses of elasto-plastic materials. We introduce a new concept from graph theory where a modeler agent is tasked with evaluating all the modeling options recast as a directed multigraph and find the optimal path that links the source of the directed graph (e.g. strain history) to the target (e.g. stress) measured by an objective function. Meanwhile, the data agent, which is tasked with generating data from real or virtual experiments (e.g. molecular dynamics, discrete …
