This chapter provides a mathematical introduction to operator surrogates, which are finite-parametric maps approximating complex (typically nonlinear) operators between infinite-dimensional (function) spaces.
A typical approach to constructing operator surrogates is the encoder-approximator-decoder paradigm, whereby infinite-dimensional inputs are mapped to a finite-dimensional latent space, processed, and then mapped back to the infinite-dimensional output space. We discuss various methods for creating efficient representation systems, including stable linear frameworks like Banach frames and data-driven techniques such as Principal Component Analysis (PCA), Active Subspaces, and nonlinear autoencoders. Subsequently, we survey key surrogate architectures, contrasting spectral methods, based on polynomial chaos expansions and related technologies, with more recent neural network approaches. We further discuss architectures that do not rely on representation systems, such as Fourier Neural Operators and transformers. We elaborate on their relationship and how they relate to other neural network operator surrogates, particularly those obtained by so-called "algorithm-unrolling".
Additionally, we provide an introduction to the theoretical analysis of approximation and generalization errors. We explain how underlying low-dimensional structures, provided for example by holomorphy, are essential to overcome the so-called "curse of dimensionality": this refers to the fact that the complexity of numerical algorithms tends to increase exponentially in the problem dimensionality. We present convergence rate bounds for holomorphic operators that depend on the number of trainable parameters and the regularity of the input and output data. Throughout, we prioritize intuition over detailed technical proofs and provide precise pointers to the literature for detailed arguments.
Physics-Informed Geometric Operators to Support Surrogate, Dimension Reduction and Generative Models for Engineering Design
BAR Technologies(BAR科技公司);Nazarbayev University(纳扎尔巴耶夫大学);University of Strathclyde(斯特拉斯克莱德大学);Foundation for Research & Technology Hellas (FORTH)(希腊研究与技术基金会);Texas A&M University(德克萨斯农工大学)
Using Deep Operators to Create Spatio-temporal Surrogates for Dynamical Systems under Uncertainty
Department of Civil and Environmental Engineering and Earth Sciences University of Notre Dame(土木与环境工程及地球科学系,诺特达姆大学);Department of Aerospace and Mechanical Engineering University of Notre Dame(航空航天与机械工程系,诺特达姆大学);Center for Research Computing University of Notre Dame(研究计算中心,诺特达姆大学)
Neural and spectral operator surrogates: unified construction and expression rate bounds
Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences(奥地利科学院约翰·拉东计算与应用数学研究所);Seminar for Applied Mathematics, ETH Zürich(苏黎世联邦理工学院应用数学研讨班);Interdisziplinäres Zentrum für wissenschaftliches Rechnen, Universität Heidelberg(海德堡大学跨学科科学计算中心)
Deep operator network for surrogate modeling of poroelasticity with random permeability fields
Department of Civil and Environmental Engineering, KAIST, Daejeon, South Korea(韩国釜山科学技术院土木与环境工程系);Department of Mathematics, North Carolina State University, Raleigh, United States(美国北卡罗来纳州立大学数学系);Department of Civil and Environmental Engineering, Seoul National University, Seoul, South Korea(韩国首尔国立大学土木与环境工程系);Institute of Construction and Environmental Engineering, Seoul National University, Seoul, South Korea(韩国首尔国立大学建设与环境工程研究所)
Residual-Based Error Corrector Operator to Enhance Accuracy and Reliability of Neural Operator Surrogates of Nonlinear Variational Boundary-Value Problems
Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin(德克萨斯大学奥斯汀分校奥登计算工程与科学研究所)
College of Computing, Georgia Institute of Technology(佐治亚理工学院计算学院);Department of Mathematics and Systems Engineering, Florida Institute of Technology(佛罗里达理工学院数学与系统工程系)
Geometry Aware Operator Transformer as an Efficient and Accurate Neural Surrogate for PDEs on Arbitrary Domains
Seminar for Applied Mathematics, ETH Zurich, Switzerland(应用数学研讨会,苏黎世联邦理工学院,瑞士);ETH AI Center, Zurich, Switzerland(ETH人工智能中心,苏黎世,瑞士);Department of Mechanical and Process Engineering, ETH Zurich, Switzerland(机械与过程工程系,苏黎世联邦理工学院,瑞士);School of Computer Science, CMU, USA(计算机科学学院,卡内基梅隆大学,美国);Centre for Applicable Mathematics, TIFR, India(应用数学中心,印度泰姬·弗里德曼研究所)
A Discrete Neural Operator with Adaptive Sampling for Surrogate Modeling of Parametric Transient Darcy Flows in Porous Media
Research Center for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong Province, 266237, China(数学与交叉科学研究中心,山东大学,青岛,山东省,266237,中国);Frontiers Science Center for Nonlinear Expectations, Minister of Education, Shandong University, Qingdao, Shandong Province, 266237, China(非线性期望前沿科学中心,教育部,山东大学,青岛,山东省,266237,中国);School of Petroleum Engineering, China University of Petroleum (East China), Qingdao, Shandong Province, 266580, China(石油工程学院,中国石油大学(华东),青岛,山东省,266580,中国);Institute of Mathematical Sciences, ShanghaiTech University, Pudong, Shanghai, 201210, China(数学科学研究院,上海科技大学,浦东,上海,201210,中国);School of Civil Engineering, Qingdao University of Technology, Qingdao, Shandong Province, 266520, China(土木工程学院,青岛科技大学,青岛,山东省,266520,中国)
Self-composing neural operators for high-frequency and multiscale PDE surrogates
Yau Mathematical Sciences Center, Tsinghua University(清华大学姚期智数学科学中心);Ocean University of China(中国海洋大学);King Abdullah University of Science and Technology(国王 Abdullah 科学与技术大学)
Neural Operator-Based Surrogate Model for CFD:Helical Coil Steam Generator in Small Modular Reactor
Department of Quantum System Engineering, Jeonbuk National University(全子系统工程系,全州国立大学);Department of Nuclear Engineering, Hanyang University(核工程系,翰阳大学);Graduate School of Integrated Energy-AI, Jeonbuk National University(整合能源-人工智能研究生院,全州国立大学)
A Fast and Generalizable Fourier Neural Operator-Based Surrogate for Melt-Pool Prediction in Laser Processing
Laboratory for Advanced Materials Processing, Swiss Federal Laboratories for Materials Science and Technology(瑞士联邦材料科学与技术实验室先进材料加工实验室);Terra Quantum AG(Terra Quantum公司)
opPINN: Physics-Informed Neural Network with operator learning to approximate solutions to the Fokker-Planck-Landau equation
Korea Institute for Advanced Study (KIAS)(韩国高等研究院);University of Southern California(南加利福尼亚大学);Pohang University of Science and Technology (POSTECH)(浦项科技大学)
Correcting Neural Operator Spectral Bias via Diffusion Posterior Sampling with Sparse Observations
Université Paris-Saclay, CentraleSupélec, CNRS, ENS Paris-Saclay(巴黎-萨克雷大学,中央理工学院,国家科学研究中心,巴黎-萨克雷理工学院);Laboratoire de Mécanique Paris-Saclay UMR 9026(巴黎-萨克雷力学实验室 UMR 9026);Politecnico di Milano(米兰理工大学);ETH AI Center(苏黎世联邦理工学院人工智能中心);Department of Mechanical Engineering University of Washington(华盛顿大学机械工程系)