We employ adaptive activation functions for regression in deep and physics-informed neural networks (PINNs) to approximate smooth and discontinuous functions as well as solutions of linear and nonlinear partial differential equations. In particular, we solve the nonlinear Klein-Gordon equation, which has smooth solutions, the nonlinear Burgers equation, which can admit high gradient solutions, and the Helmholtz equation. We introduce a scalable hyper-parameter in the activation function, which can be optimized to achieve best performance of the network as it changes dynamically the topology of the loss function involved in the optimization process. The adaptive activation function has better learning capabilities than the traditional one (fixed activation) as it improves greatly the convergence rate, especially at early training, as well as the solution accuracy. To better understand the learning process, we plot the neural network solution in the frequency domain to examine how the network captures successively different frequency bands present in the solution. We consider both forward problems, where the approximate solutions are obtained, as well as inverse problems, where parameters involved in the governing equation are identified. Our simulation results show that the proposed method is a very simple and effective approach to increase the efficiency, robustness and accuracy of the neural network approximation of nonlinear functions as well as solutions of partial differential equations, especially for forward problems.
About rectified sigmoid function for enhancing the accuracy of Physics-Informed Neural Networks
University of Nizhny Novgorod（下诺夫哥罗德国立大学）；Huawei Nizhny Novgorod Research Center（华为下诺夫哥罗德研究中心）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Data Science and Analytics, The Hong Kong University of Science and Technology (Guangzhou)（数据科学与分析，香港科学与技术大学（广州））；Internet of Things, The Hong Kong University of Science and Technology (Guangzhou)（物联网，香港科学与技术大学（广州））；Department of CEE, The Hong Kong University of Science and Technology（工程学院，香港科学与技术大学）
Physics-Informed Kolmogorov-Arnold networks for viscoelastic fluid equations
Indian Institute of Science Education and Research (IISER), Berhampur（印度科学教育与研究学院（IISER），贝尔汉普尔）；Maulana Azad National Institute of Technology (MANIT)（莫拉纳·阿扎德国立理工学院（MANIT））
DISCOVER: A Physics-Informed, GPU-Accelerated Symbolic Regression Framework
Humboldt-Universität zu Berlin（柏林洪堡大学）；Ulm University（乌尔姆大学）；Helmholtz Institute Ulm (HIU)（赫尔霍兹乌尔姆研究所）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Physics-Informed Neural Network with Squeeze-Excitation-like Attention
Key Laboratory of Quark and Lepton Physics (MOE) & Institute of Particle Physics, Central China Normal University（华中师范大学夸克与轻子物理教育部重点实验室及粒子物理研究所）；Artificial Intelligence and Computational Physics Research Center, Central China Normal University（华中师范大学人工智能与计算物理研究中心）；Key Laboratory of Nuclear Physics and Ion-beam Application (MOE) & Institute of Modern Physics, Fudan University（复旦大学核物理与离子束应用教育部重点实验室及现代物理研究所）；Shanghai Research Center for Theoretical Nuclear Physics, NSFC and Fudan University（国家自然科学基金委员会-复旦大学上海理论核物理研究中心）；Northwest Institute of Nuclear Technology（西北核技术研究所）
Analysis of three dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis
Tongji University（同济大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Beijing Institute of Technology（北京理工大学）；King Saud University（沙特国王大学）
Quantum-classical physics-informed Kolmogorov-Arnold networks for PDEs
School of Petroleum Engineering, Yangtze University（扬州大学石油工程学院）；School of Computer Science, Yangtze University（扬州大学计算机科学学院）；State Key Laboratory of Low Carbon Catalysis and Carbon Dioxide Utilization (Yangtze University)（扬州大学低碳催化与二氧化碳利用国家重点实验室）；Western Research Institute, Yangtze University（扬州大学西部研究院）
