arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.13671cs.LGcs.AI
收藏

用于偏微分方程算子学习的Transformer

Transformer for Partial Differential Equations' Operator Learning

Carnegie Mellon University(卡内基梅隆大学)

Zijie Li, Kazem Meidani, Amir Barati Farimani

AI总结:

本研究提出基于注意力的算子学习框架OFormer,以自注意力、交叉注意力和逐点MLP构建,对采样模式假设少,在标准基准上表现具竞争力且适配随机采样输入。

摘要:

偏微分方程解算子的数据驱动学习近来已成为一种有前景的近似基础解的范式。解算子通常由基于特定问题归纳偏置构建的深度学习模型进行参数化,例如利用函数值采样所在局部网格结构的卷积神经网络或图神经网络。另一方面,注意力机制提供了一种灵活的方式,可隐式挖掘输入内的模式,以及任意查询位置与输入之间的关系。本研究中,我们提出了一种用于数据驱动算子学习的基于注意力的框架,将其命名为Operator Transformer(OFormer,算子Transformer)。该框架基于自注意力、交叉注意力以及一组逐点多层感知器(MLPs)构建,因此对输入函数或查询位置的采样模式几乎不做假设。我们证明,所提出的框架在标准基准问题上具有竞争力,并且能够灵活适配随机采样的输入。

英文摘要:

Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑