arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.11907cs.LG
收藏

用于算子学习的集成与混合专家 DeepONets

Ensemble and Mixture-of-Experts DeepONets For Operator Learning

Kahlert School of Computing, University of Utah(犹他大学卡勒特计算机学院)

Ramansh Sharma, Varun Shankar

AI总结:

本文提出集成 DeepONet 与 PoU-MoE trunk 架构,证明其通用逼近性,并在二维、三维 PDE 算子学习任务中将相对 $\ell_2$ 误差降低 2–4 倍。

摘要:

我们提出了一种用于算子学习的新型深度算子网络(DeepONet)架构——集成 DeepONet,它允许用多个不同的 trunk 网络来丰富单个 DeepONet 的 trunk 网络。这种 trunk 丰富化能够在一系列算子学习问题上带来更强的表达能力和泛化能力。我们还提出了一种空间混合专家(MoE)DeepONet trunk 网络架构,该架构利用单位分解(PoU)近似来促进算子学习问题中的空间局部性和模型稀疏性。我们首先证明了集成 DeepONet 和 PoU-MoE DeepONet 都是通用逼近器。随后,我们证明,在涉及二维和三维偏微分方程(PDE)的标准算子学习问题以及具有挑战性的新算子学习问题上,包含标准 trunk、PoU-MoE trunk 和/或本征正交分解(POD)trunk 的 trunk 集成的集成 DeepONet,相比标准 DeepONet 和 POD-DeepONet 可实现低 2–4 倍的相对 $\ell_2$ 误差。我们新的 PoU-MoE 公式为将空间局部性和模型稀疏性纳入任意神经网络架构提供了自然方式,而我们新的集成 DeepONet 则为在面向算子学习的科学机器学习架构中纳入基函数丰富化提供了强大且通用的框架。

英文摘要:

We present a novel deep operator network (DeepONet) architecture for operator learning, the ensemble DeepONet, that allows for enriching the trunk network of a single DeepONet with multiple distinct trunk networks. This trunk enrichment allows for greater expressivity and generalization capabilities over a range of operator learning problems. We also present a spatial mixture-of-experts (MoE) DeepONet trunk network architecture that utilizes a partition-of-unity (PoU) approximation to promote spatial locality and model sparsity in the operator learning problem. We first prove that both the ensemble and PoU-MoE DeepONets are universal approximators. We then demonstrate that ensemble DeepONets containing a trunk ensemble of a standard trunk, the PoU-MoE trunk, and/or a proper orthogonal decomposition (POD) trunk can achieve 2-4x lower relative $\ell_2$ errors than standard DeepONets and POD-DeepONets on both standard and challenging new operator learning problems involving partial differential equations (PDEs) in two and three dimensions. Our new PoU-MoE formulation provides a natural way to incorporate spatial locality and model sparsity into any neural network architecture, while our new ensemble DeepONet provides a powerful and general framework for incorporating basis enrichment in scientific machine learning architectures for operator learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑