We introduce multi-head neural networks (MH-NNs) to physics-informed machine learning, which is a type of neural networks (NNs) with all nonlinear hidden layers as the body and multiple linear output layers as multi-head. Hence, we construct multi-head physics-informed neural networks (MH-PINNs) as a potent tool for multi-task learning (MTL), generative modeling, and few-shot learning for diverse problems in scientific machine learning (SciML). MH-PINNs connect multiple functions/tasks via a shared body as the basis functions as well as a shared distribution for the head. The former is accomplished by solving multiple tasks with MH-PINNs with each head independently corresponding to each task, while the latter by employing normalizing flows (NFs) for density estimate and generative modeling. To this end, our method is a two-stage method, and both stages can be tackled with standard deep learning tools of NNs, enabling easy implementation in practice. MH-PINNs can be used for various purposes, such as approximating stochastic processes, solving multiple tasks synergistically, providing informative prior knowledge for downstream few-shot learning tasks such as meta-learning and transfer learning, learning representative basis functions, and uncertainty quantification. We demonstrate the effectiveness of MH-PINNs in five benchmarks, investigating also the possibility of synergistic learning in regression analysis. We name the open-source code "Lernaean Hydra" (L-HYDRA), since this mythical creature possessed many heads for performing important multiple tasks, as in the proposed method.
Pacific Northwest National Laboratory（太平洋西北国家实验室）；INTERA Incorporated（INTERA公司）
Data assimilation for parameter and state estimation in subsurface transport problems remains a significant challenge due to the sparsity of measurements, the heterogeneity of porous media, and the high computational cost of forward numerical models. We present a physics-informed deep neural networks (DNNs) machine learning method for estimating space-dependent hydraulic conductivity, hydraulic head, and concentration fields from sparse measurements. In this approach, we employ individual DNNs to approximate the unknown parameters (e.g., hydraulic conductivity) and states (e.g., hydraulic head…
A Numerical Method for Coupling Parameterized Physics-Informed Neural Networks and FDM for Advanced Thermal-Hydraulic System Simulation
Division of Advanced Nuclear Engineering, Pohang University of Science and Technology（浦项科技大学先进核工程学部）；Mueunjae School of Undergraduate Studies, Pohang University of Science and Technology（浦项科技大学文在本科书院）；Department of Quantum System Engineering, Jeonbuk National University（全北国立大学量子系统工程系）
Physics-Informed Neural Networks (PINNs) offer a promising approach to solving differential equations and, more generally, to applying deep learning to problems in the physical sciences. We adopt a recently developed transfer learning approach for PINNs and introduce a multi-head model to efficiently obtain accurate solutions to nonlinear systems of ordinary differential equations with random potentials. In particular, we apply the method to simulate stochastic branched flows, a universal phenomenon in random wave dynamics. Finally, we compare the results achieved by feed forward and GAN-based…
Synthetic data generation overcomes limitations of real-world machine learning. Traditional methods are valuable for augmenting costly datasets but only optimize one criterion: realism. In this paper, we tackle the problem of generating synthetic data that optimize multiple criteria. This goal is necessary when real data are replaced by synthetic for privacy preservation. We introduce HydraGAN, a new approach to synthetic data generation that introduces multiple generator and discriminator agents into the system. The multi-agent GAN optimizes the goal of privacy-preservation as well as data re…
AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling
SVKM’s Dwarkadas J. Sanghvi College of Engineering（SVKM德瓦卡达斯·J·桑格维工程学院）；New York University Abu Dhabi (NYUAD)（纽约大学阿布扎比分校）；NYUAD Research Institute（纽约大学阿布扎比分校研究院）；The Maersk Mc-Kinney Moller Institute（马士基麦克-金尼·莫勒研究所）；University of Southern Denmark（南丹麦大学）
Sharif University of Technology（谢里夫理工大学）；Massachusetts Institute of Technology（麻省理工学院）
Physics-Informed Neural Networks (PINNs) have received increased interest for forward, inverse, and surrogate modeling of problems described by partial differential equations (PDE). However, their application to multiphysics problem, governed by several coupled PDEs, present unique challenges that have hindered the robustness and widespread applicability of this approach. Here we investigate the application of PINNs to the forward solution of problems involving thermo-hydro-mechanical (THM) processes in porous media, which exhibit disparate spatial and temporal scales in thermal conductivity, …
Nodal Hydraulic Head Estimation through Unscented Kalman Filter for Data-driven Leak Localization in Water Networks
Institut de Robòtica i Informàtica Industrial, CSIC-UPC（西班牙国家研究委员会-加泰罗尼亚理工大学工业机器人与信息技术研究所）；University of Bucharest（布加勒斯特大学）；Institute for Logic and Data Science（逻辑与数据科学研究所）；Politehnica University of Bucharest（布加勒斯特理工大学）；Supervision, Safety and Automatic Control Research Center (CS2AC) - UPC（加泰罗尼亚理工大学监督、安全与自动控制研究中心）
Physics-Informed Neural Network Modeling of Biodegradable Contaminant Transport through GCL/SL Composite Liners
Department of Civil, Environmental, and Infrastructure Engineering, George Mason University（乔治·马歇尔大学土木、环境与基础设施工程系）；State Key Laboratory of Cryospheric Science and Frozen Soil Engineering, Northwest Institute of Eco-Environment and Resources, Chinese Academy of Sciences（中国科学院寒区工程与冻土科学联合实验室，西北生态环境资源研究院）；Laboratoire Navier/CERMES, École Nationale des Ponts et Chaussées, Institut Polytechnique de Paris（巴黎理工学院劳达实验室/塞姆斯实验室，法国国家桥梁与道路学院）
MultiPUFFIN: A Multimodal Domain-Constrained Foundation Model for Molecular Property Prediction of Small Molecules
Department of Chemical Engineering, Norwegian University of Science and Technology (NTNU)（挪威科学与技术大学化学工程系）；Faculty of Industrial Engineering, KU Leuven（鲁文大学工业工程学院）；University of Surrey（萨里大学）
PhysAttNet: Enhancing Predictive Performance in Industrial and Astrophysical Time Series via Physics-Informed Attention
Faculty of Physics and Astronomy, Ruhr University Bochum（鲁尔大学波鸿分校物理与天文学学院）；Virtual Machining Group, TU Dortmund（多特蒙德工业大学虚拟加工组）；Faculty of Physics（物理学院）
Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformers
U2IS, ENSTA, Institut Polytechnique de Paris（U2IS、ENSTA、巴黎理工学院）；University of Trento（特伦托大学）；NVIDIA；University of Bergamo, Italy（意大利贝拉姆博大学）；Fondazione Bruno Kessler (FBK)（布鲁诺·凯斯勒基金会）；AMIAD, Pôle Recherche, Palaiseau（AMIAD、研究部、帕莱索）
Accurate estimates of rotation are crucial to vision-based motion estimation in augmented reality and robotics. In this work, we present a method to extract probabilistic estimates of rotation from deep regression models. First, we build on prior work and argue that a multi-headed network structure we name HydraNet provides better calibrated uncertainty estimates than methods that rely on stochastic forward passes. Second, we extend HydraNet to targets that belong to the rotation group, SO(3), by regressing unit quaternions and using the tools of rotation averaging and uncertainty injection on…
We introduce a multi-tasking graph convolutional neural network, HydraGNN, to simultaneously predict both global and atomic physical properties and demonstrate with ferromagnetic materials. We train HydraGNN on an open-source ab initio density functional theory (DFT) dataset for iron-platinum (FePt) with a fixed body centered tetragonal (BCT) lattice structure and fixed volume to simultaneously predict the mixing enthalpy (a global feature of the system), the atomic charge transfer, and the atomic magnetic moment across configurations that span the entire compositional range. By taking advanta…
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We develop a physics-informed machine learning approach for large-scale data assimilation and parameter estimation and apply it for estimating transmissivity and hydraulic head in the two-dimensional steady-state subsurface flow model of the Hanford Site given synthetic measurements of said variables. In our approach, we extend the physics-informed conditional Karhunen-Loéve expansion (PICKLE) method for modeling subsurface flow with unknown flux (Neumann) and varying head (Dirichlet) boundary conditions. We demonstrate that the PICKLE method is comparable in accuracy with the standard maximum…
Learning Dictionaries from Physical-Based Interpolation for Water Network Leak Localization
University of Bucharest（布加勒斯特大学）；Institute for Logic and Data Science（逻辑与数据科学研究所）；Institut de Robòtica i Informàtica Industrial, CSIC-UPC（西班牙国家研究委员会-加泰罗尼亚理工大学工业机器人研究所）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Politehnica University of Bucharest（布加勒斯特理工大学）
Numerical simulations of groundwater flow are used to analyze and predict the response of an aquifer system to its change in state by approximating the solution of the fundamental groundwater physical equations. The most used and classical methodologies, such as Finite Difference (FD) and Finite Element (FE) Methods, use iterative solvers which are associated with high computational cost. This study proposes a physics-based convolutional encoder-decoder neural network as a surrogate model to quickly calculate the response of the groundwater system. Holding strong promise in cross-domain mappin…
Eye Gaze-Informed and Context-Aware Pedestrian Trajectory Prediction in Shared Spaces with Automated Shuttles: A Virtual Reality Study
Department of Technology, Management and Economics at the Technical University of Denmark（丹麦技术大学技术、管理与经济学系）；Department of Transport & Planning, Civil Engineering Geosciences at Delft University of Technology（代尔夫特理工大学交通运输与规划、土木工程与地质科学系）
Multi-Scenario Highway Lane-Change Intention Prediction: A Physics-Informed AI Framework for Three-Class Classification
New York University（纽约大学）；Cornell University（康奈尔大学）；Wake Forest University（维克森林大学）；University of Pennsylvania（宾夕法尼亚大学）；Fordham University（福特汉姆大学）；Nanjing Tech University（南京工业大学）
Peking University（北京大学）；King Fahd University of Petroleum and Minerals（法赫德国王石油矿产大学）；Southern University of Science and Technology（南方科技大学）
Large-scale or high-resolution geologic models usually comprise a huge number of grid blocks, which can be computationally demanding and time-consuming to solve with numerical simulators. Therefore, it is advantageous to upscale geologic models (e.g., hydraulic conductivity) from fine-scale (high-resolution grids) to coarse-scale systems. Numerical upscaling methods have been proven to be effective and robust for coarsening geologic models, but their efficiency remains to be improved. In this work, a deep-learning-based method is proposed to upscale the fine-scale geologic models, which can as…
AI-CNet3D: An Anatomically-Informed Cross-Attention Network with Multi-Task Consistency Fine-tuning for 3D Glaucoma Classification
Department of Computer Science, Columbia University, New York, NY, USA（计算机科学系，哥伦比亚大学）；Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA（眼科学系，哥伦比亚大学伊万杰琳医疗中心）；Department of Biomedical Engineering, Columbia University, New York, NY, USA（生物医学工程系，哥伦比亚大学）；The Data Science Institute, Columbia University, New York, NY, USA（数据科学学院，哥伦比亚大学）
