Shape Derivative-Informed Neural Operators with Application to Risk-Averse Shape Optimization
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, Texas, USA（奥登计算工程与科学研究所，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯，美国）；School of Mathematical Sciences, Queensland University of Technology, Brisbane, Australia（数学科学学院，昆士兰科技大学，布里斯班，澳大利亚）；Centre for Data Science, Queensland University of Technology, Brisbane, Australia（数据科学中心，昆士兰科技大学，布里斯班，澳大利亚）；Department of Mathematics, The Ohio State University, Columbus, Ohio, USA（数学系，俄亥俄州立大学，哥伦布，俄亥俄，美国）；Department of Engineering Science and Biomedical Engineering, University of Auckland, Auckland, New Zealand（工程科学与生物医学工程系，奥克兰大学，奥克兰，新西兰）；Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, Texas, USA（沃克机械工程系，德克萨斯大学奥斯汀分校，奥斯汀，德克萨斯，美国）
Xindi Gong, Dingcheng Luo, Thomas O'Leary-Roseberry, Ruanui Nicholson, Omar Ghattas
Shape optimization under uncertainty (OUU) is computationally intensive for classical PDE-based methods due to the high cost of repeated sampling-based risk evaluation across many uncertainty realizations and varying geometries, while standard neural surrogates often fail to provide accurate and efficient sensitivities for optimization. We introduce Shape-DINO, a derivative-informed neural operator framework for learning PDE solution operators on families of varying geometries, with a particular focus on accelerating PDE-constrained shape OUU. Shape-DINOs encode geometric variability through diffeomorphic mappings to a fixed reference domain and employ a derivative-informed operator learning objective that jointly learns the PDE solution and its Fréchet derivatives with respect to design variables and uncertain parameters, enabling accurate state predictions and reliable gradients for large-scale OUU. We establish a priori error bounds linking surrogate accuracy to optimization error and prove universal approximation results for multi-input reduced basis neural operators in suitable $C^1$ norms. We demonstrate efficiency and scalability on three representative shape OUU problems, including boundary design for a Poisson equation and shape design governed by steady-state Navier-Stokes exterior flows in two and three dimensions. Across these examples, Shape-DINOs produce more reliable optimization results than operator surrogates trained without derivative information. In our examples, Shape-DINOs achieve 3-8 orders-of-magnitude speedups in state and gradient evaluations. Counting training data generation, Shape-DINOs reduce necessary PDE solves by 1-2 orders-of-magnitude compared to a strictly PDE-based approach for a single OUU problem. Moreover, Shape-DINO construction costs can be amortized across many objectives and risk measures, enabling large-scale shape OUU for complex systems.
Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning
Oden Institute for Computational Sciences and Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算科学与工程研究所）；College of Computing, Georgia Institute of Technology（佐治亚理工学院计算学院）；Walker Department of Mechanical Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校沃克机械工程系）
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
An important inference from Neural Tangent Kernel (NTK) theory is the existence of spectral bias (SB), that is, low frequency components of the target function of a fully connected Artificial Neural Network (ANN) being learnt significantly faster than the higher frequencies during training. This is established for Mean Square Error (MSE) loss functions with very low learning rate parameters. Physics Informed Neural Networks (PINNs) are designed to learn the solutions of differential equations (DE) of arbitrary orders; in PINNs the loss functions are obtained as the residues of the conservative…
Physics-Informed Neural Network (PINN) has become a commonly used machine learning approach to solve partial differential equations (PDE). But, facing high-dimensional secondorder PDE problems, PINN will suffer from severe scalability issues since its loss includes second-order derivatives, the computational cost of which will grow along with the dimension during stacked back-propagation. In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and…
LRX-PINN: A Layer-Resolving XNet Physics-Informed Neural Network with Integrated Cauchy Activations for Convection-Dominated Problems
Institute for Advanced Research, Great Bay University（先进研究院，大湾大学）；College of Computer Science and Technology, Dongguan University of Technology（计算机科学与技术学院，东莞理工大学）；Department of Mathematics, Northwestern University（数学系，西北大学）
A market with asymmetric information can be viewed as a repeated exchange game between the informed sector and the uninformed one. In a market with risk-neutral agents, De Meyer [2010] proves that the price process should be a particular kind of Brownian martingale called CMMV. This type of dynamics is due to the strategic use of their private information by the informed agents. In the current paper, we consider the more realistic case where agents on the market are risk-averse. This case is much more complex to analyze as it leads to a non-zero-sum game. Our main result is that the price proc…
This paper introduces the Hamilton-Jacobi-Bellman Proximal Policy Optimization (HJBPPO) algorithm into reinforcement learning. The Hamilton-Jacobi-Bellman (HJB) equation is used in control theory to evaluate the optimality of the value function. Our work combines the HJB equation with reinforcement learning in continuous state and action spaces to improve the training of the value network. We treat the value network as a Physics-Informed Neural Network (PINN) to solve for the HJB equation by computing its derivatives with respect to its inputs exactly. The Proximal Policy Optimization (PPO)-Cl…
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
organization= Department of Mathematics, Institut Teknologi Sepuluh Nopember , city= Surabaya , postcode= 60111 , country= Indonesia；organization= Department of Mathematics, Khalifa University of Science \& Technology , city= Abu Dhabi , postcode= PO Box 127788 , country= United Arab Emirates；organization= Department of Mathematics, Faculty of Mathematics
This work is concerned with discovering the governing partial differential equation (PDE) of a physical system. Existing methods have demonstrated the PDE identification from finite observations but failed to maintain satisfying results against noisy data, partly owing to suboptimal estimated derivatives and found PDE coefficients. We address the issues by introducing a noise-aware physics-informed machine learning (nPIML) framework to discover the governing PDE from data following arbitrary distributions. We propose training a couple of neural networks, namely solver and preselector, in a mul…
SRM Institute of Science and Technology（SRM科学技术研究所）；University of Utah（犹他大学）
We present a new technique for the accelerated training of physics-informed neural networks (PINNs): discretely-trained PINNs (DT-PINNs). The repeated computation of partial derivative terms in the PINN loss functions via automatic differentiation during training is known to be computationally expensive, especially for higher-order derivatives. DT-PINNs are trained by replacing these exact spatial derivatives with high-order accurate numerical discretizations computed using meshless radial basis function-finite differences (RBF-FD) and applied via sparse-matrix vector multiplication. The use o…
University of California at Santa Barbara（加州大学圣巴巴拉分校）；Massachusetts Institute of Technology（麻省理工学院）；New York University Grossman School of Medicine（纽约大学格罗斯曼医学院）
Electrical properties (EP), namely permittivity and electric conductivity, dictate the interactions between electromagnetic waves and biological tissue. EP can be potential biomarkers for pathology characterization, such as cancer, and improve therapeutic modalities, such radiofrequency hyperthermia and ablation. MR-based electrical properties tomography (MR-EPT) uses MR measurements to reconstruct the EP maps. Using the homogeneous Helmholtz equation, EP can be directly computed through calculations of second order spatial derivatives of the measured magnetic transmit or receive fields $(B_{1…
The University of Arizona（亚利桑那大学）；Massachusetts Institute of Technology（麻省理工学院）
This study presents a novel unsupervised convolutional Neural Network (NN) architecture with nonlocal interactions for solving Partial Differential Equations (PDEs). The nonlocal Peridynamic Differential Operator (PDDO) is employed as a convolutional filter for evaluating derivatives the field variable. The NN captures the time-dynamics in smaller latent space through encoder-decoder layers with a Convolutional Long-short Term Memory (ConvLSTM) layer between them. The ConvLSTM architecture is modified by employing a novel activation function to improve the predictive capability of the learning…
Virginia Tech（弗吉尼亚理工大学）；University of Michigan（密歇根大学）
Physics-informed Neural Networks (PINNs) are gaining attention in the engineering and scientific literature for solving a range of differential equations with applications in weather modeling, healthcare, manufacturing, etc. Poor scalability is one of the barriers to utilizing PINNs for many real-world problems. To address this, a Self-scalable tanh (Stan) activation function is proposed for the PINNs. The proposed Stan function is smooth, non-saturating, and has a trainable parameter. During training, it can allow easy flow of gradients to compute the required derivatives and also enable syst…
Physics-Informed Neural Networks (PINNs) have become a kind of attractive machine learning method for obtaining solutions of partial differential equations (PDEs). Training PINNs can be seen as a semi-supervised learning task, in which only exact values of initial and boundary points can be obtained in solving forward problems, and in the whole spatio-temporal domain collocation points are sampled without exact labels, which brings training difficulties. Thus the selection of collocation points and sampling methods are quite crucial in training PINNs. Existing sampling methods include fixed an…
FO-PINNs: A First-Order formulation for Physics Informed Neural Networks
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）；University of California Davis（加州大学戴维斯分校）；Illinois Institute of Technology（伊利诺伊理工学院）
Physics-informed neural networks have been widely applied to learn general parametric solutions of differential equations. Here, we propose a neural network to discover parametric eigenvalue and eigenfunction surfaces of quantum systems. We apply our method to solve the hydrogen molecular ion. This is an ab-initio deep learning method that solves the Schrodinger equation with the Coulomb potential yielding realistic wavefunctions that include a cusp at the ion positions. The neural solutions are continuous and differentiable functions of the interatomic distance and their derivatives are analy…
Fourier Feature Pyramids for Physics-Informed Neural Networks
Department of Computing and Mathematical Sciences, California Institute of Technology（计算与数学科学部，加州理工学院）；Cahill Center for Astronomy and Astrophysics, California Institute of Technology（天文与天体物理中心，加州理工学院）；Google DeepMind（谷歌DeepMind）
We consider the approximation of a class of dynamic partial differential equations (PDE) of second order in time by the physics-informed neural network (PINN) approach, and provide an error analysis of PINN for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation. Our analyses show that, with feed-forward neural networks having two hidden layers and the $\tanh$ activation function, the PINN approximation errors for the solution field, its time derivative and its gradient field can be effectively bounded by the training loss and the number of training data points (q…
Institute of Informatics（信息学研究所）；Ludwig Maximilians University Munich（慕尼黑路德维希-马克西米利安大学）；NASA Goddard Space Flight Center（美国国家航空航天局戈达德太空飞行中心）
Physics-informed Neural Networks (PINNs) often have, in their loss functions, terms based on physical equations and derivatives. In order to evaluate these terms, the output solution is sampled using a distribution of collocation points. However, density-based strategies, in which the number of collocation points over the domain increases throughout the training period, do not scale well to multiple spatial dimensions. To remedy this issue, we present here a curriculum-training-based method for lightweight collocation point distributions during network training. We apply this method to a PINN …
In many computational problems in engineering and science, function or model differentiation is essential, but also integration is needed. An important class of computational problems include so-called integro-differential equations which include both integrals and derivatives of a function. In another example, stochastic differential equations can be written in terms of a partial differential equation of a probability density function of the stochastic variable. To learn characteristics of the stochastic variable based on the density function, specific integral transforms, namely moments, of …
Advanced Physics-Informed Neural Network with Residuals for Solving Complex Integral Equations
Department of Computer and Data Sciences, Shahid Beheshti University（计算机与数据科学系，谢赫·贝赫什提大学）；Department of Cognitive Modeling, Shahid Beheshti University（认知建模系，谢赫·贝赫什提大学）
Technical University of Darmstadt（达姆施塔特工业大学）；RWTH Aachen University（亚琛工业大学）
Physics-informed neural networks (PINNs) are capable of finding the solution for a given boundary value problem. We employ several ideas from the finite element method (FEM) to enhance the performance of existing PINNs in engineering problems. The main contribution of the current work is to promote using the spatial gradient of the primary variable as an output from separated neural networks. Later on, the strong form which has a higher order of derivatives is applied to the spatial gradients of the primary variable as the physical constraint. In addition, the so-called energy form of the prob…
Purdue University（普渡大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）
Physics-informed neural networks (PINNs) have been widely used to solve partial differential equations in a forward and inverse manner using deep neural networks. However, training these networks can be challenging for multiscale problems. While statistical methods can be employed to scale the regression loss on data, it is generally challenging to scale the loss terms for equations. This paper proposes a method for scaling the mean squared loss terms in the objective function used to train PINNs. Instead of using automatic differentiation to calculate the temporal derivative, we use backward …
SPINONet: Scalable Spiking Physics-informed Neural Operator for Computational Mechanics Applications
Department of Applied Mechanics（应用力学系）；Indian Institute of Technology Delhi（德里印度理工学院）；Institute of Structural Mechanics and Dynamics in Aerospace Engineering（航空航天工程中的结构力学与动力学研究所）；Faculty of Aerospace Engineering and Geodesy（航空航天工程与大地测量学系）；University of Stuttgart（斯图加特大学）；Experimental Hepatobiliary Surgery Group（肝胆外科实验小组）；Department of Hepatobiliary Surgery and Visceral Transplantation（肝胆外科与内脏移植部）；University of Leipzig Medical Center（莱比锡大学医学中心）；Department of Civil and Systems Engineering（土木与系统工程系）；Johns Hopkins University（约翰霍普金斯大学）；Data Science and AI Institute（数据科学与人工智能研究所）；Yardi School of Artificial Intelligence (YScAI)（Yardi人工智能学院（YScAI））
