arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2201.01032cs.LGcs.AIphysics.comp-ph
收藏

基于耦合注意的算子学习

Learning Operators with Coupled Attention

University of Pennsylvania(宾夕法尼亚大学)

Georgios Kissas, Jacob Seidman, Leonardo Ferreira Guilhoto, Victor M. Preciado, George J. Pappas, Paris Perdikaris

摘要:

有监督算子学习是一种新兴的机器学习范式,其应用于时空动态系统的演化建模和函数数据之间的一般黑箱关系的近似。我们提出了一种新的算子学习方法,LOCA(学习算子耦合注意),最近的成功的注意力机制的动机。在我们的架构中,输入功能映射到一个有限的功能,然后平均与注意力的权重,取决于输出查询的位置。通过将这些注意力权重与积分变换耦合在一起,LOCA能够显式地学习目标输出函数中的相关性,即使在训练集测量中的输出函数的数量非常小时,我们也能够近似非线性算子。我们的配方是伴随着严格的近似理论保证所提出的模型的普遍表现力。从经验上讲,我们评估了LOCA在几个操作员学习场景中的性能,这些场景涉及由常微分方程和偏微分方程控制的系统,以及黑箱气候预测问题。通过这些场景,我们展示了最先进的准确性,相对于噪声输入数据的鲁棒性,以及在测试数据集上始终较小的误差分布,即使是分布外的预测任务。

英文摘要:

Supervised operator learning is an emerging machine learning paradigm with applications to modeling the evolution of spatio-temporal dynamical systems and approximating general black-box relationships between functional data. We propose a novel operator learning method, LOCA (Learning Operators with Coupled Attention), motivated from the recent success of the attention mechanism. In our architecture, the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions, enabling us to approximate nonlinear operators even when the number of output function in the training set measurements is very small. Our formulation is accompanied by rigorous approximation theoretic guarantees on the universal expressiveness of the proposed model. Empirically, we evaluate the performance of LOCA on several operator learning scenarios involving systems governed by ordinary and partial differential equations, as well as a black-box climate prediction problem. Through these scenarios we demonstrate state of the art accuracy, robustness with respect to noisy input data, and a consistently small spread of errors over testing data sets, even for out-of-distribution prediction tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑