Training Variation of Physically-Informed Deep Learning Models
Department of Materials Science and Engineering（材料科学与工程系）；The Ohio State University（俄亥俄州立大学）；Center for Integrated Nanotechnologies（集成纳米技术中心）；Sandia National Laboratories（桑迪亚国家实验室）；BlueQuartz Software（BlueQuartz软件公司）；Department of Mechanical and Aerospace Engineering（机械与航空航天工程系）
A successful deep learning network is highly dependent not only on the training dataset, but the training algorithm used to condition the network for a given task. The loss function, dataset, and tuning of hyperparameters all play an essential role in training a network, yet there is not much discussion on the reliability or reproducibility of a training algorithm. With the rise in popularity of physics-informed loss functions, this raises the question of how reliable one's loss function is in conditioning a network to enforce a particular boundary condition. Reporting the model variation is needed to assess a loss function's ability to consistently train a network to obey a given boundary condition, and provides a fairer comparison among different methods. In this work, a Pix2Pix network predicting the stress fields of high elastic contrast composites is used as a case study. Several different loss functions enforcing stress equilibrium are implemented, with each displaying different levels of variation in convergence, accuracy, and enforcing stress equilibrium across many training sessions. Suggested practices in reporting model variation are also shared.
Inverse source problems are central to many applications in acoustics, geophysics, non-destructive testing, and more. Traditional imaging methods suffer from the resolution limit, preventing distinction of sources separated by less than the emitted wavelength. In this work we propose a method based on physically-informed neural-networks for solving the source refocusing problem, constructing a novel loss term which promotes super-resolving capabilities of the network and is based on the physics of wave propagation. We demonstrate the approach in the setup of imaging an a-priori unknown number …
ENS Paris-Saclay（巴黎-萨克雷高等师范学院）；Mines Paris（巴黎矿业学院）；Institut de Radioprotection et de Sûreté Nucléaire（法国辐射防护与核安全研究所）
Temperature dependence of the neutron-nucleus interaction is known as the Doppler broadening of the cross-sections. This is a well-known effect due to the thermal motion of the target nuclei that occurs in the neutron-nucleus interaction. The fast computation of such effects is crucial for any nuclear application. Mechanisms have been developed that allow determining the Doppler effects in the cross-section, most of them based on the numerical resolution of the equation known as Solbrig's kernel, which is a cross-section Doppler broadening formalism derived from a free gas atoms distribution h…
Aragon Institute of Engineering Research (I3A)（阿拉贡工程研究院（I3A））；University of Zaragoza（萨拉戈萨大学）；Higher Technical School of Industrial Engineering, Polytechnic University of Madrid（马德里理工大学高等工业工程学院）；ESI Group（ESI集团）；ENSAM Institute of Technology（法国国立高等工程技术学院）
The imminent impact of immersive technologies in society urges for active research in real-time and interactive physics simulation for virtual worlds to be realistic. In this context, realistic means to be compliant to the laws of physics. In this paper we present a method for computing the dynamic response of (possibly non-linear and dissipative) deformable objects induced by real-time user interactions in mixed reality using deep learning. The graph-based architecture of the method ensures the thermodynamic consistency of the predictions, whereas the visualization pipeline allows a natural a…
Reactivity-Informed Machine Learning for Performance Prediction and Design Space Exploration of Alkali-Activated Slag
Department of Civil and Environmental Engineering, Rice University（Rice大学土木与环境工程系）；Rice Advanced Materials Institute, Rice University（Rice大学先进材料研究所）；Ken Kennedy Institute, Rice University（Rice大学肯尼迪研究所）
Q-GAIN: A Python Package for Machine Learning and Physically Informed Analysis Applications
National Institute of Standards and Technology（国家标准与技术研究院）；Joint Quantum Institute and University of Maryland（联合量子研究所和马里兰大学）；Department of Physics and Astronomy（物理与天文学系）；Joint Center for Quantum Information and Computer Science（联合量子信息与计算机科学中心）
Physically-informed change-point kernels for structural dynamics
Dynamics Research Group, Department of Mechanical Engineering, University of Sheffield, Mappin Street, Sheffield S1 3JD, United Kingdom（谢菲尔德大学机械工程系动力学研究组）
Indian Institute of Technology, Gandhinagar（印度理工学院甘地讷格尔分校）
Current modeling approaches for hydrological modeling often rely on either physics-based or data-science methods, including Machine Learning (ML) algorithms. While physics-based models tend to rigid structure resulting in unrealistic parameter values in certain instances, ML algorithms establish the input-output relationship while ignoring the constraints imposed by well-known physical processes. While there is a notion that the physics model enables better process understanding and ML algorithms exhibit better predictive skills, scientific knowledge that does not add to predictive ability may…
Synergizing Physically Constrained MCMC and Chemical-Informed Gaussian Processes for Reaction Network Discovery
State Key Laboratory of Fine Chemicals, Frontier Science Center for Smart Materials, Dalian University of Technology, Dalian, 116024, China（精细化学品国家重点实验室，智能材料前沿科学中心，大连理工大学，大连，116024，中国）；Dalian Key Laboratory of Intelligent Chemistry, CR Belt and Road Joint Laboratory on Intelligent Chemistry and Advanced Materials of Liaoning Province, School of Chemistry, Dalian University of Technology, Dalian, 116024, China（大连智能化学重点实验室，辽宁省“一带一路”智能化学与先进材料联合实验室，化学学院，大连理工大学，大连，116024，中国）；Academy of Interdisciplinary Studies, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong（交叉学科研究院，香港理工大学，清水湾，九龙，香港）
Why Physics Still Matters: Improving Machine Learning Prediction of Material Properties with Phonon-Informed Datasets
Department of Physics, Universitat Politècnica de Catalunya, 08034 Barcelona, Spain（物理系，巴塞罗那理工大学）；Research Center in Multiscale Science（多尺度科学研究中心）；Department of Electronic Engineering, Universitat Politècnica de Catalunya, 08034 Barcelona, Spain（电子工程系，巴塞罗那理工大学）；Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan（工业科学研究所，东京大学）
Physics-informed neural network for predicting fatigue life of unirradiated and irradiated austenitic and ferritic/martensitic steels under reactor-relevant conditions
Extreme Environments Materials Group (EEMG), Department of Materials Engineering, Indian Institute of Science (IISc)（极端环境材料组（EEMG）、材料工程系、印度科学研究院（IISc））；Department of Materials, University of Oxford（材料系、牛津大学）；Robert Bosch Centre for Cyberphysical Systems, Indian Institute of Science (IISc)（罗伯特·博世网络物理系统中心、印度科学研究院（IISc））
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We propose a new class of physics-informed neural networks, called physics-informed Variational Autoencoder (PI-VAE), to solve stochastic differential equations (SDEs) or inverse problems involving SDEs. In these problems the governing equations are known but only a limited number of measurements of system parameters are available. PI-VAE consists of a variational autoencoder (VAE), which generates samples of system variables and parameters. This generative model is integrated with the governing equations. In this integration, the derivatives of VAE outputs are readily calculated using automat…
Learning Physics-Informed Surrogate Model of Linear Elastic Displacement Fields from Geometry
PROMES-CNRS, Université Perpignan Via Domitia (UPVD)（法国国家科学研究中心佩皮尼昂大学多姆itia分校能源与过程系统优化实验室）；LAAS-CNRS, Université de Toulouse (UT)（法国国家科学研究中心图卢兹大学航空航天系统实验室）
Physics-Informed Neural Networks (PINNs) offer a promising approach to solving differential equations and, more generally, to applying deep learning to problems in the physical sciences. We adopt a recently developed transfer learning approach for PINNs and introduce a multi-head model to efficiently obtain accurate solutions to nonlinear systems of ordinary differential equations with random potentials. In particular, we apply the method to simulate stochastic branched flows, a universal phenomenon in random wave dynamics. Finally, we compare the results achieved by feed forward and GAN-based…
Brown University（布朗大学）；Worcester Polytechnic Institute（伍斯特理工学院）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
We formulate a general framework for hp-variational physics-informed neural networks (hp-VPINNs) based on the nonlinear approximation of shallow and deep neural networks and hp-refinement via domain decomposition and projection onto space of high-order polynomials. The trial space is the space of neural network, which is defined globally over the whole computational domain, while the test space contains the piecewise polynomials. Specifically in this study, the hp-refinement corresponds to a global approximation with local learning algorithm that can efficiently localize the network parameter …
In this work we analyze how quadrature rules of different precisions and piecewise polynomial test functions of different degrees affect the convergence rate of Variational Physics Informed Neural Networks (VPINN) with respect to mesh refinement, while solving elliptic boundary-value problems. Using a Petrov-Galerkin framework relying on an inf-sup condition, we derive an a priori error estimate in the energy norm between the exact solution and a suitable high-order piecewise interpolant of a computed neural network. Numerical experiments confirm the theoretical predictions and highlight the i…
Scalable Quantum Error Mitigation with Physically Informed Graph Neural Networks
National Supercomputing Center in Zhengzhou（郑州国家超级计算中心）；School of Computer and Artificial Intelligence（计算机与人工智能学院）；School of Communication and Artificial Intelligence（通信与人工智能学院）；School of Integrated Circuits（集成电路学院）；Nanjing Institute of Technology（南京理工大学）
Brown University（布朗大学）；Worcester Polytechnic Institute（伍斯特理工学院）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the \textit{variational residual} of the PDE using the DNN approximation by incorporating the variational form o…
In a finite element analysis, using a large number of grids is important to obtain accurate results, but is a resource-consuming task. Aiming to real-time simulation and optimization, it is desired to obtain fine grid analysis results within a limited resource. This paper proposes a super-resolution method that predicts a stress tensor field in a high-resolution from low-resolution contour plots by utilizing a U-Net-based neural network which is called PI-UNet. In addition, the proposed model minimizes the residual of the equilibrium constraints so that it outputs a physically reasonable solut…
California Institute of Technology（加州理工学院）；Purdue University（普渡大学）
We introduce a scheme for probabilistic hypocenter inversion with Stein variational inference. Our approach uses a differentiable forward model in the form of a physics informed neural network, which we train to solve the Eikonal equation. This allows for rapid approximation of the posterior by iteratively optimizing a collection of particles against a kernelized Stein discrepancy. We show that the method is well-equipped to handle highly multimodal posterior distributions, which are common in hypocentral inverse problems. A suite of experiments is performed to examine the influence of the var…
Imperial College London（帝国理工学院）；Microsoft Research（微软研究院）；University College London（伦敦大学学院）
Learning workable representations of dynamical systems is becoming an increasingly important problem in a number of application areas. By leveraging recent work connecting deep neural networks to systems of differential equations, we propose \emph{variational integrator networks}, a class of neural network architectures designed to preserve the geometric structure of physical systems. This class of network architectures facilitates accurate long-term prediction, interpretability, and data-efficient learning, while still remaining highly flexible and capable of modeling complex behavior. We dem…
Dynamical systems are found in innumerable forms across the physical and biological sciences, yet all these systems fall naturally into universal equivalence classes: conservative or dissipative, stable or unstable, compressible or incompressible. Predicting these classes from data remains an essential open challenge in computational physics at which existing time-series classification methods struggle. Here, we propose, \texttt{phase2vec}, an embedding method that learns high-quality, physically-meaningful representations of 2D dynamical systems without supervision. Our embeddings are produce…
INI-VPINN: A Variational Physics-Informed Neural Network with Implicit Neumann and Interface Handling for Multi-Material Domains with Geometric Singularities
Variational Quantum Physics-Informed Neural Networks for Hydrological PDE-Constrained Learning with Inherent Uncertainty Quantification
Faculty of Computer Science and Multimedia, Lincoln University College, Petaling Jaya, Selangor, Malaysia（计算机科学与多媒体学院，林肯大学学院，Petaling Jaya，Selangor，马来西亚）
Physically Interpretable Representation Learning with Gaussian Mixture Variational AutoEncoder (GM-VAE)
Institute for Computational and Mathematical Engineering, Stanford University（计算与数学工程研究所，斯坦福大学）；Department of Mechanical Engineering, Stanford University（机械工程系，斯坦福大学）；National Renewable Energy Laboratory（国家可再生能源实验室）；Ann and H.J. Smead Department of Aerospace Engineering Sciences, University of Colorado, Boulder（安与H.J.斯迈德航空航天工程科学系，科罗拉多大学，博尔德）
As applications of deep learning (DL) continue to seep into critical scientific use-cases, the importance of performing uncertainty quantification (UQ) with DL has become more pressing than ever before. In scientific applications, it is also important to inform the learning of DL models with knowledge of physics of the problem to produce physically consistent and generalized solutions. This is referred to as the emerging field of physics-informed deep learning (PIDL). We consider the problem of developing PIDL formulations that can also perform UQ. To this end, we propose a novel physics-infor…
Swiss Federal Laboratories for Materials Science and Technology (Empa)（瑞士联邦材料科学与技术实验室（Empa））；Swiss Federal Institute of Technology Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
Due to their high energy intensity, buildings play a major role in the current worldwide energy transition. Building models are ubiquitous since they are needed at each stage of the life of buildings, i.e. for design, retrofitting, and control operations. Classical white-box models, based on physical equations, are bound to follow the laws of physics but the specific design of their underlying structure might hinder their expressiveness and hence their accuracy. On the other hand, black-box models are better suited to capture nonlinear building dynamics and thus can often achieve better accura…
A Primer on Variational Inference for Physics-Informed Deep Generative Modelling
University of Cambridge（剑桥大学）；Imperial College London（伦敦帝国理工学院）；London School of Economics and Political Science（伦敦政治经济学院）；The Alan Turing Institute（阿兰·图灵研究所）
