Meta-materials are an important emerging class of engineered materials in which complex macroscopic behaviour--whether electromagnetic, thermal, or mechanical--arises from modular substructure. Simulation and optimization of these materials are computationally challenging, as rich substructures necessitate high-fidelity finite element meshes to solve the governing PDEs. To address this, we leverage parametric modular structure to learn component-level surrogates, enabling cheaper high-fidelity simulation. We use a neural network to model the stored potential energy in a component given boundary conditions. This yields a structured prediction task: macroscopic behavior is determined by the minimizer of the system's total potential energy, which can be approximated by composing these surrogate models. Composable energy surrogates thus permit simulation in the reduced basis of component boundaries. Costly ground-truth simulation of the full structure is avoided, as training data are generated by performing finite element analysis with individual components. Using dataset aggregation to choose training boundary conditions allows us to learn energy surrogates which produce accurate macroscopic behavior when composed, accelerating simulation of parametric meta-materials.
Numerical simulations for engineering applications solve partial differential equations (PDE) to model various physical processes. Traditional PDE solvers are very accurate but computationally costly. On the other hand, Machine Learning (ML) methods offer a significant computational speedup but face challenges with accuracy and generalization to different PDE conditions, such as geometry, boundary conditions, initial conditions and PDE source terms. In this work, we propose a novel ML-based approach, CoAE-MLSim (Composable AutoEncoder Machine Learning Simulation), which is an unsupervised, low…
In this paper we show that our Machine Learning (ML) approach, CoMLSim (Composable Machine Learning Simulator), can simulate PDEs on highly-resolved grids with higher accuracy and generalization to out-of-distribution source terms and geometries than traditional ML baselines. Our unique approach combines key principles of traditional PDE solvers with local-learning and low-dimensional manifold techniques to iteratively simulate PDEs on large computational domains. The proposed approach is validated on more than 5 steady-state PDEs across different PDE conditions on highly-resolved grids and co…
Linear Partial Differential Equations (PDEs) govern the spatial-temporal dynamics of physical systems that are essential to building modern technology. When working with linear PDEs, designing a physical system for a specific outcome is difficult and costly due to slow and expensive explicit simulation of PDEs and the highly nonlinear relationship between a system's configuration and its behavior. In this work, we prove a parametric form that certain physical quantities in the Fourier domain must obey in linear PDEs, named the CZP (Constant-Zeros-Poles) framework. Applying CZP to antenna desig…
University of Warwick（华威大学）；The Alan Turing Institute（艾伦·图灵研究所）
Specifying a governing physical model in the presence of missing physics and recovering its parameters are two intertwined and fundamental problems in science. Modern machine learning allows one to circumvent these, via emulators and surrogates, but in doing so disregards prior knowledge and physical laws that are especially important for small data regimes, interpretability, and decision making. In this work we fold the mechanistic model into a flexible data-driven surrogate to arrive at a physically structured decoder network. This provides accelerated inference for the Bayesian inverse prob…
SINTEF（挪威科学与工业研究基金会）；ETH Zürich（苏黎世联邦理工学院）；University of Southern California（南加州大学）；TIFR Centre for Applicable Mathematics（塔塔基础研究院应用数学中心）
We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Under suitable hypotheses, we show that the resulting optimizers converge exponentially fast (and with exponentially decaying variance), with respect to increasing number of training sample…
Out-of-distribution generalization of deep-learning surrogates for 2D PDE-generated dynamics in the small-data regime
Institute for Advanced Simulations – Materials Data Science and Informatics (IAS-9), Forschungszentrum Jülich GmbH（先进模拟研究所——材料数据科学与信息学研究所（IAS-9））；Chair of Materials Data Science and Materials Informatics, Faculty 5 – Georesources and Materials Engineering, RWTH Aachen University（材料数据科学与材料信息学教授，地质资源与材料工程系，亚琛RWTH大学）
MIT（麻省理工学院）；IBM Research AI（IBM研究院人工智能部门）；IBM Thomas J Watson Research Center（IBM托马斯·J·沃森研究院）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；Iowa State University（爱荷华州立大学）
Surrogate models for partial-differential equations are widely used in the design of meta-materials to rapidly evaluate the behavior of composable components. However, the training cost of accurate surrogates by machine learning can rapidly increase with the number of variables. For photonic-device models, we find that this training becomes especially challenging as design regions grow larger than the optical wavelength. We present an active learning algorithm that reduces the number of training points by more than an order of magnitude for a neural-network surrogate model of optical-surface c…
College of Computing, Georgia Institute of Technology（佐治亚理工学院计算学院）；Department of Mathematics and Systems Engineering, Florida Institute of Technology（佛罗里达理工学院数学与系统工程系）
Fast Dynamic 1D Simulation of Divertor Plasmas with Neural PDE Surrogates
Eindhoven University of Technology（埃因霍温理工大学）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Dutch Institute for Fundamental Energy Research（荷兰基础能源研究所）；Forschungszentrum Jülich GmbH（于利希研究中心）
We investigate a deep learning approach to efficiently perform Bayesian inference in partial differential equation (PDE) and integral equation models over potentially high-dimensional parameter spaces. The contributions of this paper are two-fold; the first is the introduction of a neural network approach to approximating the solutions of Fredholm and Volterra integral equations of the first and second kind. The second is the development of a new, efficient deep learning-based method for Bayesian inversion applied to problems that can be described by PDEs or integral equations. To achieve this…
University of Notre Dame（圣母大学）；Technical University of Munich（慕尼黑工业大学）；University of Pennsylvania（宾夕法尼亚大学）
Surrogate modeling and uncertainty quantification tasks for PDE systems are most often considered as supervised learning problems where input and output data pairs are used for training. The construction of such emulators is by definition a small data problem which poses challenges to deep learning approaches that have been developed to operate in the big data regime. Even in cases where such models have been shown to have good predictive capability in high dimensions, they fail to address constraints in the data implied by the PDE model. This paper provides a methodology that incorporates the…
Gradient-Informed Temporal Sampling Improves Rollout Accuracy in PDE Surrogate Training
School of Future Technology, South China University of Technology, China（未来技术学院，华南理工大学，中国）；State Key Laboratory of Ocean Sensing & Ocean College, Zhejiang University, China（海洋感知国家重点实验室及海洋学院，浙江大学，中国）；Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology, USA（天体物理与空间研究所，麻省理工学院，美国）
Numerical simulations in climate, chemistry, or astrophysics are computationally too expensive for uncertainty quantification or parameter-exploration at high-resolution. Reduced-order or surrogate models are multiple orders of magnitude faster, but traditional surrogates are inflexible or inaccurate and pure machine learning (ML)-based surrogates too data-hungry. We propose a hybrid, flexible surrogate model that exploits known physics for simulating large-scale dynamics and limits learning to the hard-to-model term, which is called parametrization or closure and captures the effect of fine- …
Self-composing neural operators for high-frequency and multiscale PDE surrogates
Yau Mathematical Sciences Center, Tsinghua University（清华大学姚期智数学科学中心）；Ocean University of China（中国海洋大学）；King Abdullah University of Science and Technology（国王 Abdullah 科学与技术大学）
Microsoft Autonomous Systems and Robotics Research（微软自主系统与机器人研究部）；Microsoft Research AI4Science（微软研究院AI4Science）
Partial differential equations (PDEs) are central to describing complex physical system simulations. Their expensive solution techniques have led to an increased interest in deep neural network based surrogates. However, the practical utility of training such surrogates is contingent on their ability to model complex multi-scale spatio-temporal phenomena. Various neural network architectures have been proposed to target such phenomena, most notably Fourier Neural Operators (FNOs), which give a natural handle over local & global spatial information via parameterization of different Fourier mode…
University of Stuttgart（斯图加特大学）；SimTech（模拟技术中心）；IMPRS-IS（国际马克斯·普朗克研究学院智能系统研究所）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL University（巴黎文理研究大学）；NEC Labs Europe（日本电气欧洲实验室）
Microsoft Research AI4Science（微软研究院AI4Science）；Microsoft Autonomous Systems and Robotics Research（微软自主系统与机器人研究院）
Partial differential equations (PDEs) see widespread use in sciences and engineering to describe simulation of physical processes as scalar and vector fields interacting and coevolving over time. Due to the computationally expensive nature of their standard solution methods, neural PDE surrogates have become an active research topic to accelerate these simulations. However, current methods do not explicitly take into account the relationship between different fields and their internal components, which are often correlated. Viewing the time evolution of such correlated fields through the lens …
Accelerating Simulation of Two-Phase Flows with Neural PDE Surrogates
Swiss Plasma Center, École Polytechnique Fédérale de Lausanne（瑞士等离子体中心，洛桑联邦理工学院）；Data and Artificial Intelligence Cluster, Eindhoven University of Technology（埃因霍温理工大学数据与人工智能集群）；Centre for Analysis, Scientific Computing, and Applications, Eindhoven University of Technology（埃因霍温理工大学分析、科学计算与应用中心）
Advanced Computing, Mathematics and Data Division, Pacific Northwest National Laboratory（太平洋西北国家实验室先进计算、数学和数据 division）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；Atmospheric Sciences and Global Change Division, Pacific Northwest National Laboratory（太平洋西北国家实验室大气科学与全球变化 division）
