We address the solution of large-scale Bayesian optimal experimental design (OED) problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields. The OED problem seeks to find sensor locations that maximize the expected information gain (EIG) in the solution of the underlying Bayesian inverse problem. Computation of the EIG is usually prohibitive for PDE-based OED problems. To make the evaluation of the EIG tractable, we approximate the (PDE-based) parameter-to-observable map with a derivative-informed projected neural network (DIPNet) surrogate, which exploits the geometry, smoothness, and intrinsic low-dimensionality of the map using a small and dimension-independent number of PDE solves. The surrogate is then deployed within a greedy algorithm-based solution of the OED problem such that no further PDE solves are required. We analyze the EIG approximation error in terms of the generalization error of the DIPNet and show they are of the same order. Finally, the efficiency and accuracy of the method are demonstrated via numerical experiments on OED problems governed by inverse scattering and inverse reactive transport with up to 16,641 uncertain parameters and 100 experimental design variables, where we observe up to three orders of magnitude speedup relative to a reference double loop Monte Carlo method.
Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning
Oden Institute for Computational Sciences and Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算科学与工程研究所）；College of Computing, Georgia Institute of Technology（佐治亚理工学院计算学院）；Walker Department of Mechanical Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校沃克机械工程系）
Oden Institute for Computational Engineering & Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学学院）；McKelvey School of Engineering, Washington University in St. Louis（圣路易斯华盛顿大学麦凯维工程学院）；Department of Mechanical Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校机械工程系）
Many-query problems, arising from uncertainty quantification, Bayesian inversion, Bayesian optimal experimental design, and optimization under uncertainty-require numerous evaluations of a parameter-to-output map. These evaluations become prohibitive if this parametric map is high-dimensional and involves expensive solution of partial differential equations (PDEs). To tackle this challenge, we propose to construct surrogates for high-dimensional PDE-governed parametric maps in the form of projected neural networks that parsimoniously capture the geometry and intrinsic low-dimensionality of the…
Dimension reduction for derivative-informed operator learning: An analysis of approximation errors
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学研究所）；School of Computational Science and Engineering, Georgia Institute of Technology（佐治亚理工学院计算科学与工程学院）；Walker Department of Mechanical Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校沃克机械工程系）
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
Shape Derivative-Informed Neural Operators with Application to Risk-Averse Shape Optimization
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas, USA（奥登计算工程与科学研究所，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯，美国）；School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia（数学科学学院，昆士兰科技大学，布里斯班，澳大利亚）；Centre for Data Science, Queensland University of Technology, Brisbane, Australia（数据科学中心，昆士兰科技大学，布里斯班，澳大利亚）；Department of Mathematics, The Ohio State University, Columbus, Ohio, USA（数学系，俄亥俄州立大学，哥伦布，俄亥俄，美国）；Department of Engineering Science and Biomedical Engineering, University of Auckland, Auckland, New Zealand（工程科学与生物医学工程系，奥克兰大学，奥克兰，新西兰）；Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA（沃克机械工程系，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯，美国）
Derivative-informed Graph Convolutional Autoencoder with Phase Classification for the Lifshitz-Petrich Model
Department of Mathematics, University of Massachusetts Dartmouth（达特茅斯大学数学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学数学科学学院）；School of Mathematical Sciences, CMA-Shanghai and MOE-LSC, Shanghai Jiao Tong University（上海交通大学数学科学学院）
An important inference from Neural Tangent Kernel (NTK) theory is the existence of spectral bias (SB), that is, low frequency components of the target function of a fully connected Artificial Neural Network (ANN) being learnt significantly faster than the higher frequencies during training. This is established for Mean Square Error (MSE) loss functions with very low learning rate parameters. Physics Informed Neural Networks (PINNs) are designed to learn the solutions of differential equations (DE) of arbitrary orders; in PINNs the loss functions are obtained as the residues of the conservative…
Physics-informed reduced order model with conditional neural fields
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Notre Dame（圣母大学）；Arizona State University（亚利桑那州立大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）
Fractional partial differential equations (FPDEs) can effectively represent anomalous transport and nonlocal interactions. However, inherent uncertainties arise naturally in real applications due to random forcing or unknown material properties. Mathematical models considering nonlocal interactions with uncertainty quantification can be formulated as stochastic fractional partial differential equations (SFPDEs). There are many challenges in solving SFPDEs numerically, especially for long-time integration since such problems are high-dimensional and nonlocal. Here, we combine the bi-orthogonal …
Learning on the Temporal Tangent Bundle for Physics-Informed Neural Networks
École Nationale Supérieure de Génie Mathématique et Modélisation (ENSGMM), Université Nationale des Sciences, Technologies, Ingénierie et Mathématiques (UNSTIM)（国家科学、技术、工程与数学大学）；Département de Mathématiques (DM), Université d’Abomey-Calavi (UAC)（阿波美-卡拉维大学）；Institut de Mathématiques et de Sciences Physiques (IMSP), Université d’Abomey-Calavi (UAC)（阿波美-卡拉维大学）
This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimality of the value function. Our work combines the HJB equation with reinforcement learning in continuous state and action spaces to improve the training of the value network. We treat the value network as a Physics-Informed Neural Network (PINN) to solve for the HJB equation by computing its derivatives with respect to its inputs exactly. The Proximal Policy Optimization (PPO)-Cl…
This work is concerned with discovering the governing partial differential equation (PDE) of a physical system. Existing methods have demonstrated the PDE identification from finite observations but failed to maintain satisfying results against noisy data, partly owing to suboptimal estimated derivatives and found PDE coefficients. We address the issues by introducing a noise-aware physics-informed machine learning (nPIML) framework to discover the governing PDE from data following arbitrary distributions. We propose training a couple of neural networks, namely solver and preselector, in a mul…
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
organization= Department of Mathematics, Institut Teknologi Sepuluh Nopember , city= Surabaya , postcode= 60111 , country= Indonesia；organization= Department of Mathematics, Khalifa University of Science \& Technology , city= Abu Dhabi , postcode= PO Box 127788 , country= United Arab Emirates；organization= Department of Mathematics, Faculty of Mathematics
University of California at Santa Barbara（加州大学圣巴巴拉分校）；Massachusetts Institute of Technology（麻省理工学院）；New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）
Electrical properties (EP), namely permittivity and electric conductivity, dictate the interactions between electromagnetic waves and biological tissue. EP can be potential biomarkers for pathology characterization, such as cancer, and improve therapeutic modalities, such radiofrequency hyperthermia and ablation. MR-based electrical properties tomography (MR-EPT) uses MR measurements to reconstruct the EP maps. Using the homogeneous Helmholtz equation, EP can be directly computed through calculations of second order spatial derivatives of the measured magnetic transmit or receive fields $(B_{1…
Projected Sobolev Natural Gradient Descent for Efficient Neural Network Solution of the Gross--Pitaevskii Equation
Yau Mathematical Sciences Center, Tsinghua University（丘成桐数学科学中心，清华大学）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（燕京量子信息科学与应用研究院）；Hunan Key Laboratory for Computation and Simulation in Science and Engineering, Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education, School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）
Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivatives
Hong Kong University of Science and Technology（香港科技大学）；University of Maryland College Park（马里兰大学帕克分校）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（港科大深港协同创新研究院）
A novel refinement measure for non-intrusive surrogate modelling of partial differential equations (PDEs) with uncertain parameters is proposed. Our approach uses an empirical interpolation procedure, where the proposed refinement measure is based on a PDE residual and probability density function of the uncertain parameters, and excludes parts of the PDE solution that are not used to compute the quantity of interest. The PDE residual used in the refinement measure is computed by using all the partial derivatives that enter the PDE separately. The proposed refinement measure is suited for effi…
Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees
RIKEN（理化学研究所）；Interdisciplinary Theoretical and Mathematical Sciences Program (iTHEMS)（跨学科理论与数学科学项目（iTHEMS））；Center for Quantum Computing, RIKEN（理化学研究所量子计算中心）
The University of Arizona（亚利桑那大学）；Massachusetts Institute of Technology（麻省理工学院）
This study presents a novel unsupervised convolutional Neural Network (NN) architecture with nonlocal interactions for solving Partial Differential Equations (PDEs). The nonlocal Peridynamic Differential Operator (PDDO) is employed as a convolutional filter for evaluating derivatives the field variable. The NN captures the time-dynamics in smaller latent space through encoder-decoder layers with a Convolutional Long-short Term Memory (ConvLSTM) layer between them. The ConvLSTM architecture is modified by employing a novel activation function to improve the predictive capability of the learning…
Physics Informed Neural Networks (PINNs) is a promising application of deep learning. The smooth architecture of a fully connected neural network is appropriate for finding the solutions of PDEs; the corresponding loss function can also be intuitively designed and guarantees the convergence for various kinds of PDEs. However, the rate of convergence has been considered as a weakness of this approach. This paper proposes Sobolev-PINNs, a novel loss function for the training of PINNs, making the training substantially efficient. Inspired by the recent studies that incorporate derivative informat…
Pointwise Error Estimates for Numerical Physics-Informed Neural Networks
School of Computer Science, Georgia Institute of Technology（佐治亚理工学院计算机学院）；CentraleSupélec, Paris Saclay University（巴黎萨克雷大学中央理工-高等电力学院）；Transvalor SA（Transvalor公司）
We consider the approximation of a class of dynamic partial differential equations (PDE) of second order in time by the physics-informed neural network (PINN) approach, and provide an error analysis of PINN for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation. Our analyses show that, with feed-forward neural networks having two hidden layers and the $\tanh$ activation function, the PINN approximation errors for the solution field, its time derivative and its gradient field can be effectively bounded by the training loss and the number of training data points (q…
Physics-Informed Neural Networks (PINNs) have become a kind of attractive machine learning method for obtaining solutions of partial differential equations (PDEs). Training PINNs can be seen as a semi-supervised learning task, in which only exact values of initial and boundary points can be obtained in solving forward problems, and in the whole spatio-temporal domain collocation points are sampled without exact labels, which brings training difficulties. Thus the selection of collocation points and sampling methods are quite crucial in training PINNs. Existing sampling methods include fixed an…
FO-PINNs: A First-Order formulation for Physics Informed Neural Networks
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）；University of California Davis（加州大学戴维斯分校）；Illinois Institute of Technology（伊利诺伊理工学院）
