Complex dynamical systems are used for predictions in many domains. Because of computational costs, models are truncated, coarsened, or aggregated. As the neglected and unresolved terms become important, the utility of model predictions diminishes. We develop a novel, versatile, and rigorous methodology to learn non-Markovian closure parameterizations for known-physics/low-fidelity models using data from high-fidelity simulations. The new "neural closure models" augment low-fidelity models with neural delay differential equations (nDDEs), motivated by the Mori-Zwanzig formulation and the inherent delays in complex dynamical systems. We demonstrate that neural closures efficiently account for truncated modes in reduced-order-models, capture the effects of subgrid-scale processes in coarse models, and augment the simplification of complex biological and physical-biogeochemical models. We find that using non-Markovian over Markovian closures improves long-term prediction accuracy and requires smaller networks. We derive adjoint equations and network architectures needed to efficiently implement the new discrete and distributed nDDEs, for any time-integration schemes and allowing nonuniformly-spaced temporal training data. The performance of discrete over distributed delays in closure models is explained using information theory, and we find an optimal amount of past information for a specified architecture. Finally, we analyze computational complexity and explain the limited additional cost due to neural closure models.
Oklahoma State University（俄克拉荷马州立大学）；Norwegian University of Science and Technology（挪威科技大学）；SINTEF Digital（SINTEF数字研究所）
A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement lea…
Structure-preserving neural networks for the regularized entropy-based closure of the Boltzmann moment system
Computer Science and Mathematics Division（计算机科学与数学分部）；Oak Ridge National Laboratory（橡树岭国家实验室）；Faculty of Mathematics（数学学院）；KIT（卡尔斯鲁厄理工学院）；University of Tennessee（田纳西大学）
The Gestalt laws of perceptual organization, which describe how visual elements in an image are grouped and interpreted, have traditionally been thought of as innate despite their ecological validity. We use deep-learning methods to investigate whether natural scene statistics might be sufficient to derive the Gestalt laws. We examine the law of closure, which asserts that human visual perception tends to "close the gap" by assembling elements that can jointly be interpreted as a complete figure or object. We demonstrate that a state-of-the-art convolutional neural network, trained to classify…
A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networks
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin（德克萨斯大学奥斯汀分校奥登计算工程与科学学院）；Computational Science Center, National Renewable Energy Laboratory（美国国家可再生能源实验室计算科学中心）
Energy-Conserving Neural Network Closure Model for Long-Time Accurate and Stable LES
Centrum Wiskunde \& Informatica, Science Park 123, Amsterdam, The Netherlands；Centre for Analysis, Scientific Computing；Applications, Eindhoven University of Technology, PO Box 513,Eindhoven, 5600 MB,The Netherlands
Quantifying Out-of-Training Uncertainty of Neural-Network based Turbulence Closures
Mechanical and Aerospace Engineering, Utah State University（犹他州立大学机械与航空航天工程学院）；Idaho National Laboratory（爱达荷国家实验室）；School of Nuclear Science and Engineering, Oregon State University（俄勒冈州立大学核科学与工程学院）
Differentiable Physics-Neural Models enable Learning of Non-Markovian Closures for Accelerated Coarse-Grained Physics Simulations
Department of Mechanical Engineering（机械工程系）；National University of Singapore（新加坡国立大学）；Institute of High Performance Computing（高性能计算研究所）；Agency for Science, Technology and Research（科学、技术与研究局）；Dept of Mechanical and Aerospace Engineering（机械与航空航天工程系）；Nanyang Technological University（南洋理工大学）
Bias Inheritance in Neural-Symbolic Discovery of Constitutive Closures Under Function-Class Mismatch
Nanophotonics and Biophotonics Key Laboratory of Jilin Province, School of Physics, Changchun University of Science and Technology（吉林省纳米光子学与生物光子学重点实验室，物理学院，长春理工大学）
Comparison of neural closure models for discretised PDEs
Eindhoven University of Technology（埃因霍温理工大学）；Centrum Wiskunde & Informatica（荷兰数学与计算机科学研究院）；Korteweg-de Vries Institute for Mathematics, University of Amsterdam（阿姆斯特丹大学科特韦格-德弗里斯数学研究所）
In this article, we prove approximation theorems in classes of deep and shallow neural networks with analytic activation functions by elementary arguments. We prove for both real and complex networks with non-polynomial activation that the closure of the class of neural networks coincides with the closure of the space of polynomials. The closure can further be characterized by the Stone-Weierstrass theorem (in the real case) and Mergelyan's theorem (in the complex case). In the real case, we further prove approximation results for networks with higher-dimensional harmonic activation and orthog…
Coarse Graining with Neural Operators for Simulating Chaotic Systems
Department of Computing and Mathematical Sciences, California Institute of Technology（计算与数学科学系，加利福尼亚理工学院）；NVIDIA Research（NVIDIA研究）；Graduate Aerospace Laboratories, California Institute of Technology（航空航天实验室，加利福尼亚理工学院）
Electron neural closure for turbulent magnetosheath simulations: energy channels
Centre for mathematical Plasma Astrophysics, Department of Mathematics, KU Leuven（数学等离子体天文学中心，数学系，KU莱顿大学）；Department of Physics, University of Wisconsin-Madison（物理系，威斯康星大学麦迪逊分校）
The Machine Learning Approach to Moment Closure Relations for Plasma: A Review
School of Physical and Chemical Sciences, Queen Mary University of London（伦敦大学女王学院物理与化学科学学院）；Space Weather TREC, University of Colorado（科罗拉多大学空间天气TREC）
We trained deep neural networks (DNNs) as a function of the neutrino energy density, flux, and the fluid velocity to reproduce the Eddington tensor for neutrinos obtained in our first-principles core-collapse supernova (CCSN) simulations. Although the moment method, which is one of the most popular approximations for neutrino transport, requires a closure relation, none of the analytical closure relations commonly employed in the literature captures all aspects of the neutrino angular distribution in momentum space. In this paper, we developed a closure relation by using the DNN that takes the…
Calibration of a neural network ocean closure for improved mean state and variability
Courant Institute School of Mathematics, Computing, and Data Science, New York University, New York, NY, USA（科朗学院数学、计算与数据科学学院，纽约大学，纽约，纽约州，美国）；Program in Atmospheric and Oceanic Sciences, Princeton University, Princeton, NJ, USA（大气与海洋科学项目，普林斯顿大学，普林斯顿，新泽西州，美国）
Robust Moment Identification for Nonlinear PDEs via a Neural ODE Approach
Department of Mathematics and Statistics, University of Massachusetts Amherst（数学与统计学系，马萨诸塞大学阿默斯特分校）；School of Mathematics, Georgia Institute of Technology（数学学院，佐治亚理工学院）
