arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.14376cs.LGcs.NAmath.NAphysics.comp-ph
收藏

GNOT:一种用于算子学习的通用神经算子Transformer

GNOT: A General Neural Operator Transformer for Operator Learning

Tsinghua University(清华大学);RealAI;Bosch China Investment Ltd(博世中国投资有限公司)

Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu

AI总结:

针对算子学习中非规则网格、多输入等挑战,提出基于Transformer的通用神经算子框架GNOT,结合异质归一化注意力层与几何门控机制,在多领域数据集上实现性能显著提升。

摘要:

学习偏微分方程(PDEs)的解算子是机器学习中的一个核心问题。然而,在实际应用中,算子学习面临若干挑战,例如非规则网格、多输入函数以及PDE解的复杂性。为应对这些挑战,我们提出了通用神经算子Transformer(GNOT),这是一种可扩展且高效的基于Transformer的算子学习框架。通过设计一种新型的异质归一化注意力层,我们的模型在处理多输入函数和非规则网格时具备高度灵活性。此外,我们引入了一种几何门控机制,可将其视为一种软区域分解方法,用于解决多尺度问题。Transformer架构的大模型容量使我们的模型能够扩展到大型数据集和实际问题中。我们在来自不同领域的多个具有挑战性的数据集上开展了大量实验,与现有替代方法相比,取得了显著的性能提升。我们的代码和数据已在https://github.com/thu-ml/GNOT 公开发布。

英文摘要:

Learning partial differential equations' (PDEs) solution operators is an essential problem in machine learning. However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution. To address these challenges, we propose a general neural operator transformer (GNOT), a scalable and effective transformer-based framework for learning operators. By designing a novel heterogeneous normalized attention layer, our model is highly flexible to handle multiple input functions and irregular meshes. Besides, we introduce a geometric gating mechanism which could be viewed as a soft domain decomposition to solve the multi-scale problems. The large model capacity of the transformer architecture grants our model the possibility to scale to large datasets and practical problems. We conduct extensive experiments on multiple challenging datasets from different domains and achieve a remarkable improvement compared with alternative methods. Our code and data are publicly available at \url{https://github.com/thu-ml/GNOT}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑