Transformer作为有限正则性微分方程解的神经算子
Transformers as Neural Operators for Solutions of Differential Equations with Finite Regularity
AI总结:
本研究建立Transformer作为神经算子的通用逼近理论,并应用于有限正则性微分方程解预测,在多个动力系统上验证,相比DeepONet精度更高但计算更昂贵。
摘要:
神经算子学习模型已成为计算科学与工程中不同应用场景下偏微分方程(PDE)数据驱动方法中非常有效的替代模型。此类算子学习模型不仅能实时预测物理或生物系统的特定实例,还能预测对应于初始条件、边界条件或强迫项分布的一类解。DeepONet是第一个神经算子模型,并已针对包括黎曼问题在内的广泛解类型进行了广泛测试。Transformer尚未被用于此能力,特别是尚未针对低正则性PDE解进行测试。在本工作中,我们首先建立理论基础,证明Transformer作为算子学习模型具有通用逼近性质。然后,我们将Transformer应用于预测具有有限正则性解的各种动力系统的解,针对多种初始条件和强迫项。具体而言,我们考虑三个示例:Izhikevich神经元模型、调温分数阶漏积分放电(LIF)模型以及一维欧拉方程黎曼问题。对于后一个问题,我们还与DeepONet的变体进行比较,发现Transformer在精度上优于DeepONet,但计算成本更高。
英文摘要:
Neural operator learning models have emerged as very effective surrogates in data-driven methods for partial differential equations (PDEs) across different applications from computational science and engineering. Such operator learning models not only predict particular instances of a physical or biological system in real-time but also forecast classes of solutions corresponding to a distribution of initial and boundary conditions or forcing terms. % DeepONet is the first neural operator model and has been tested extensively for a broad class of solutions, including Riemann problems. Transformers have not been used in that capacity, and specifically, they have not been tested for solutions of PDEs with low regularity. % In this work, we first establish the theoretical groundwork that transformers possess the universal approximation property as operator learning models. We then apply transformers to forecast solutions of diverse dynamical systems with solutions of finite regularity for a plurality of initial conditions and forcing terms. In particular, we consider three examples: the Izhikevich neuron model, the tempered fractional-order Leaky Integrate-and-Fire (LIF) model, and the one-dimensional Euler equation Riemann problem. For the latter problem, we also compare with variants of DeepONet, and we find that transformers outperform DeepONet in accuracy but they are computationally more expensive.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。