Physics-Informed Neural Network (PINN) has become a commonly used machine learning approach to solve partial differential equations (PDE). But, facing high-dimensional secondorder PDE problems, PINN will suffer from severe scalability issues since its loss includes second-order derivatives, the computational cost of which will grow along with the dimension during stacked back-propagation. In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. We further discuss the model capacity and provide variance reduction methods to address key limitations in the derivative estimation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is significantly faster. Our code is released at https://github.com/LithiumDA/PINN-without-Stacked-BP.
Nanjing Audit University（南京审计大学）；Southwestern University of Finance and Economics（西南财经大学）
Forecasting stock market direction is always an amazing but challenging problem in finance. Although many popular shallow computational methods (such as Backpropagation Network and Support Vector Machine) have extensively been proposed, most algorithms have not yet attained a desirable level of applicability. In this paper, we present a deep learning model with strong ability to generate high level feature representations for accurate financial prediction. Precisely, a stacked denoising autoencoder (SDAE) from deep learning is applied to predict the daily CSI 300 index, from Shanghai and Shenz…
FastLRNR and Sparse Physics Informed Backpropagation
Telepix；Department of Computer Science, Arizona State University（亚利桑那州立大学计算机科学系）；School of Computing, Korea Advanced Institute of Science and Technology（韩国科学技术院计算机科学学院）；Department of Mathematics, Washington University in St. Louis（圣路易斯华盛顿大学数学系）；Department of Mathematics, University of Central Florida（中央佛罗里达大学数学系）
Lagged backward-compatible physics-informed neural networks for unsaturated soil consolidation analysis
George Mason University（乔治梅森大学）；National Institute of Natural Hazards, Ministry of Emergency Management（应急管理部国家自然灾害防治研究院）；Northwest Institute of Eco-Environment and Resources, Chinese Academy of Sciences（中国科学院西北生态环境资源研究院）；École Nationale des Ponts et Chaussées（巴黎路桥学院）；Hebei University of Engineering（河北工程大学）；Zhejiang University of Technology（浙江工业大学）
Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations
School of Automation, Central South University（中南大学自动化学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；China Institute of Water Resources and Hydropower Research（中国水利水电科学研究院）
Micro-bubbles and bubbly flows are widely observed and applied in chemical engineering, medicine, involves deformation, rupture, and collision of bubbles, phase mixture, etc. We study bubble dynamics by setting up two numerical simulation cases: bubbly flow with a single bubble and multiple bubbles, both confined in the microchannel, with parameters corresponding to their medical backgrounds. Both the cases have their medical background applications. Multiphase flow simulation requires high computation accuracy due to possible component losses that may be caused by sparse meshing during the co…
Time-dependent Partial Differential Equations with given initial conditions are considered in this paper. New differentiation techniques of the unknown solution with respect to time variable are proposed. It is shown that the proposed techniques allow to generate accurate higher order derivatives simultaneously for a set of spatial points. The calculated derivatives can then be used for data-driven solution in different ways. An application for Physics Informed Neural Networks by the well-known DeepXDE software solution in Python under Tensorflow background framework has been presented for thr…
This work presents a physics-informed neural network (PINN) based framework to model the strain-rate and temperature dependence of the deformation fields in elastic-viscoplastic solids. To avoid unbalanced back-propagated gradients during training, the proposed framework uses a simple strategy with no added computational complexity for selecting scalar weights that balance the interplay between different terms in the physics-based loss function. In addition, we highlight a fundamental challenge involving the selection of appropriate model outputs so that the mechanical problem can be faithfull…
Munich Data Science Institute（慕尼黑数据科学研究所）；School of Computation, Information and Technology, Technical University of Munich（技术大学慕尼黑计算、信息与技术学院）；Interdisciplinary Center for Scientific Computing (IWR)（跨学科科学计算中心）；Department of Mathematics and Computer Science, Heidelberg University（海德堡大学数学与计算机科学系）
Short-term traffic forecasting based on deep learning methods, especially recurrent neural networks (RNN), has received much attention in recent years. However, the potential of RNN-based models in traffic forecasting has not yet been fully exploited in terms of the predictive power of spatial-temporal data and the capability of handling missing data. In this paper, we focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models. A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assi…
Physics-informed neural networks (PINN) have recently emerged as a promising application of deep learning in a wide range of engineering and scientific problems based on partial differential equation (PDE) models. However, evidence shows that PINN training by gradient descent displays pathologies that often prevent convergence when solving PDEs with irregular solutions. In this paper, we propose the use of a particle swarm optimization (PSO) approach to train PINNs. The resulting PSO-PINN algorithm not only mitigates the undesired behaviors of PINNs trained with standard gradient descent but a…
We apply Physics-Informed Neural Networks (PINNs) for solving identification problems of nonhomogeneous materials. We focus on the problem with a background in elasticity imaging, where one seeks to identify the nonhomogeneous mechanical properties of soft tissue based on the full-field displacement measurements under quasi-static loading. In our model, we apply two independent neural networks, one for approximating the solution of the corresponding forward problem, and the other for approximating the unknown material parameter field. As a proof of concept, we validate our model on a prototypi…
As an emerging technology in deep learning, physics-informed neural networks (PINNs) have been widely used to solve various partial differential equations (PDEs) in engineering. However, PDEs based on practical considerations contain multiple physical quantities and complex initial boundary conditions, thus PINNs often returns incorrect results. Here we take heat transfer problem in multilayer fabrics as a typical example. It is coupled by multiple temperature fields with strong correlation, and the values of variables are extremely unbalanced among different dimensions. We clarify the potenti…
Northeastern University（东北大学）；Sichuan University（四川大学）；University of Notre Dame（圣母大学）；Renmin University of China（中国人民大学）
High-fidelity simulation of complex physical systems is exorbitantly expensive and inaccessible across spatiotemporal scales. Recently, there has been an increasing interest in leveraging deep learning to augment scientific data based on the coarse-grained simulations, which is of cheap computational expense and retains satisfactory solution accuracy. However, the major existing work focuses on data-driven approaches which rely on rich training datasets and lack sufficient physical constraints. To this end, we propose a novel and efficient spatiotemporal super-resolution framework via physics-…
Universidade Federal do Rio de Janeiro（里约热内卢联邦大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
With the increasing amount of available data and advances in computing capabilities, deep neural networks (DNNs) have been successfully employed to solve challenging tasks in various areas, including healthcare, climate, and finance. Nevertheless, state-of-the-art DNNs are susceptible to quasi-imperceptible perturbed versions of the original images -- adversarial examples. These perturbations of the network input can lead to disastrous implications in critical areas where wrong decisions can directly affect human lives. Adversarial training is the most efficient solution to defend the network …
Multi-Scenario Highway Lane-Change Intention Prediction: A Physics-Informed AI Framework for Three-Class Classification
New York University（纽约大学）；Cornell University（康奈尔大学）；Wake Forest University（维克森林大学）；University of Pennsylvania（宾夕法尼亚大学）；Fordham University（福特汉姆大学）；Nanjing Tech University（南京工业大学）
Neural networks can be used to learn the solution of partial differential equations (PDEs) on arbitrary domains without requiring a computational mesh. Common approaches integrate differential operators in training neural networks using a structured loss function. The most common training algorithm for neural networks is backpropagation which relies on the gradient of the loss function with respect to the parameters of the network. In this work, we characterize the difficulty of training neural networks on physics by investigating the impact of differential operators in corrupting the back pro…
Adversarial Disentanglement by Backpropagation with Physics-Informed Variational Autoencoder
Department of Civil Engineering and Geosciences（土木工程与地球科学系）；Technical University of Delft（代尔夫特理工大学）；Department of Engineering（工程系）；University of Cambridge（剑桥大学）
Physics-informed network paradigm with data generation and background noise removal for diverse distributed acoustic sensing applications
State Key Laboratory of Advanced Optical Communication Systems and Networks, Department of Electronic Engineering, Shanghai Jiao Tong University（先进光通信系统与网络国家重点实验室，电子工程学院，上海交通大学）
Gradient-Informed Temporal Sampling Improves Rollout Accuracy in PDE Surrogate Training
School of Future Technology, South China University of Technology, China（未来技术学院，华南理工大学，中国）；State Key Laboratory of Ocean Sensing & Ocean College, Zhejiang University, China（海洋感知国家重点实验室及海洋学院，浙江大学，中国）；Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology, USA（天体物理与空间研究所，麻省理工学院，美国）
Linear Stability Analysis of Physics-Informed Random Projection Neural Networks for ODEs
Modelling Engineering Risk and Complexity, Scuola Superiore Meridionale（工程风险与复杂性建模，南方高级学院）；Dept. of Electrical and Computer Engineering, Clarkson University（电气与计算机工程系，克林顿大学）；Department of Statistics, Athens University of Economics and Business（统计学系，雅典经济与商业大学）
