Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for integrating physics-based constraints and data to address forward and inverse problems in machine learning. Despite their potential, the implementation of PINNs are hampered by several challenges, including issues related to convergence, stability, and the design of neural networks and loss functions. In this paper, we introduce a novel training scheme that addresses these challenges by framing the training process as a constrained optimization problem. Utilizing a quadratic program (QP)-based gradient descent law, our approach simplifies the design of loss functions and guarantees convergences to optimal neural network parameters. This methodology enables dynamic balancing, over the course of training, between data-based loss and a partial differential equation (PDE) residual loss, ensuring an acceptable level of accuracy while prioritizing the minimization of PDE-based loss. We demonstrate the formulation of the constrained PINNs approach with noisy data, in the context of solving Laplace's equation in a capacitor with complex geometry. This work not only advances the capabilities of PINNs but also provides a framework for their training.
Modeling of an ODE-constrained optimization problem describing tumor dynamics, and numerical approximation via sequential physics-informed neural networks
Chebyshev Center-Based Direction Selection for Multi-Objective Optimization and Training PINNs
Department of Industrial & Systems Engineering, KAIST(韩国科学技术院工业与系统工程系);Discipline of Business Analytics, The University of Sydney(悉尼大学商业分析学科);Department of Mathematical Sciences, Seoul National University(首尔国立大学数学科学系)
A Trust Region Method for Finding Second-Order Stationarity in Linearly Constrained Non-Convex Optimization
受TRACE算法[Curtis et al. 2017]的启发,我们提出了一种用于寻找线性约束非凸优化问题的二阶稳定点的置信域算法。证明了该算法在\widetilde{\mathcal{O}}(\max{ε_g^{-3/2}, ε_H^{-3}})迭代下收敛到(ε_g,ε_H)-二阶稳定点.这种迭代复杂性是针对一般线性约束优化而实现的,而无需对目标函数进行立方正则化。
Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm
organization= Department of Mathematics, Institut Teknologi Sepuluh Nopember , city= Surabaya , postcode= 60111 , country= Indonesia;organization= Department of Mathematics, Khalifa University of Science \& Technology , city= Abu Dhabi , postcode= PO Box 127788 , country= United Arab Emirates;organization= Department of Mathematics, Faculty of Mathematics