本文首次为深度算子网络建立了数据依赖型规模下界,证明在 n 个数据点上获得低训练误差需要公共输出维度按 Ω(√[4]{n}) 增长,并通过求解对流-扩散-反应 PDE 的实验表明训练数据规模可能需随该维度至少二次方增长。
摘要:
深度算子网络(Deep Operator Networks)是一种日益流行的范式,用于求解无穷维空间中的回归问题,从而一次性求解偏微分方程族。在本工作中,我们旨在建立首个数据依赖型的下界,用以刻画深度算子网络(DeepONets)在噪声数据上降低经验误差所需的最小规模。特别地,我们证明:若要在 n 个数据点上获得较低的训练误差,分支网络(branch net)与主干网络(trunk net)的公共输出维度必须按 Ω(√[4]{n}) 的规模增长。这一结果启发了我们使用深度算子网络求解对流-扩散-反应偏微分方程的实验。在实验中,我们展示了这样一种可能性:在固定模型规模下,若要利用公共输出维度的增加来获得训练误差的单调下降,训练数据规模可能至少需要随该维度呈二次方增长。
英文摘要:
Deep Operator Networks are an increasingly popular paradigm for solving regression in infinite dimensions and hence solve families of PDEs in one shot. In this work, we aim to establish a first-of-its-kind data-dependent lowerbound on the size of DeepONets required for them to be able to reduce empirical error on noisy data. In particular, we show that for low training errors to be obtained on $n$ data points it is necessary that the common output dimension of the branch and the trunk net be scaling as $Ω\left ( \sqrt[\leftroot{-1}\uproot{-1}4]{n} \right )$.
This inspires our experiments with DeepONets solving the advection-diffusion-reaction PDE, where we demonstrate the possibility that at a fixed model size, to leverage increase in this common output dimension and get monotonic lowering of training error, the size of the training data might necessarily need to scale at least quadratically with it.
Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problems
Pacific Northwest National Laboratory(太平洋西北国家实验室);Sandia National Laboratories(桑迪亚国家实验室);Division of Applied Mathematics and School of Engineering, Brown University(布朗大学应用数学系与工程学院)
Deep set based operator learning with uncertainty quantification
Department of Mathematics, Shanghai Normal University(上海师范大学数学系);School of Mathematical Sciences, Institute of Natural Sciences(自然科学院数学系);MOE-LSC, Shanghai Jiao Tong University(教育部重点实验室,上海交通大学);Institute of Computational Mathematics(计算数学研究所);Scientific/Engineering Computing, AMSS, Chinese Academy of Sciences(科学与工程计算,中国科学院数学研究所)
Deep Parallel Spectral Neural Operators for Solving Partial Differential Equations with Enhanced Low-Frequency Learning Capability
College of Computer Science and Technology(计算机科学与技术学院);China University of Petroleum (East China)(中国石油大学(华东));State Key Laboratory of Chemical Safety(化学安全国家重点实验室)
Deep learning methods for inverse problems using connections between proximal operators and Hamilton-Jacobi equations
Industrial and Systems Engineering, Lehigh University(工业与系统工程,莱维大学);Department of Mathematics, University of Illinois Urbana-Champaign(数学系,伊利诺伊大学厄巴纳-香槟分校)
fPINN-DeepONet: A Physics-Informed Operator Learning Framework for Multi-term Time-fractional Mixed Diffusion-wave Equations
Department of Mathematics, Purdue University West Lafayette, IN 47907, USA(普渡大学数学系);Department of Mathematics, Purdue University School of Mechanical Engineering, Purdue University West Lafayette, IN 47907, USA(普渡大学机械工程学院数学系)
Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport
University of Pennsylvania(宾夕法尼亚大学);Massachusetts Institute of Technology(麻省理工学院)
Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method
Toulouse-INP (ENSEEIHT)/IRIT(图卢兹INP(ENSEEIHT)/IRIT);Artificial and Natural Intelligence Toulouse Institute (ANITI)(人工智能与自然智能图卢兹研究所(ANITI));Division of Applied Mathematics, Brown University(应用数学系,布朗大学)
Deep Ridgelet Transform: Voice with Koopman Operator Proves Universality of Formal Deep Networks
Center for Advanced Intelligence Project (AIP), RIKEN(RIKEN 先进智能项目中心);NTT Network Service Systems Laboratories, NTT Corporation(NTT 网络服务系统实验室,NTT 公司);Center for Data Science, Ehime University(爱媛大学数据科学中心)
On Approximating the Dynamic Response of Synchronous Generators via Operator Learning: A Step Towards Building Deep Operator-based Power Grid Simulators
MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problems
A DeepONet for inverting the Neumann-to-Dirichlet Operator in Electrical Impedance Tomography: An approximation theoretic perspective and numerical results
Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve University(数学与应用数学及统计学系,凯斯西储大学);School of AI and Advanced Computing, Xi’an Jiaotong Liverpool University(人工智能与先进计算学院,西安交通大学利物浦大学)
Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study
Department of Mathematics, Hong Kong Baptist University(香港 Baptist 大学数学系);Department of ACMS, University of Notre Dame(Notre Dame 大学ACMS系);School of Mathematics, Georgia Institute of Technology(佐治亚理工学院数学系);Department of Mathematics, UCLA(加州大学洛杉矶分校数学系)
Structure and Distribution Metric for Quantifying the Quality of Uncertainty: Assessing Gaussian Processes, Deep Neural Nets, and Deep Neural Operators for Regression
Deep Learning with Kernels through RKHM and the Perron-Frobenius Operator
NTT Network Service Systems Laboratories(NTT网络服务系统研究所);RIKEN AIP(理化学研究所革新智能统合研究中心);Keio University(庆应义塾大学);Aix-Marseille University(艾克斯-马赛大学);CNRS(法国国家科学研究中心);LIS(信息科学实验室)