Nonlocal Attention Operator:面向可解释物理发现的隐藏知识具象化
Nonlocal Attention Operator: Materializing Hidden Knowledge Towards Interpretable Physics Discovery
AI总结:
针对物理系统学习中PDE反问题的不适定性,提出基于注意力机制的非局部注意力算子NAO,证明其等价于带数据依赖核的双重积分算子,可提取全局先验、提升泛化性,为可解释物理基础模型学习和注意力机制理解提供新方向。
摘要:
尽管基于注意力的神经架构近期在自然语言处理(NLP)和计算机视觉(CV)等核心AI领域广受关注,但其在复杂物理系统建模中的潜力仍未得到充分探索。物理系统中的学习问题通常被刻画为基于少量函数对实例,发现能在函数空间之间进行映射的算子。这类任务往往呈现为严重不适定的偏微分方程(PDE)反问题。本研究提出一种基于注意力机制的新型神经算子架构,我们将其命名为Nonlocal Attention Operator(NAO,非局部注意力算子),并探索其构建基础物理模型的能力。具体而言,我们证明注意力机制等价于一种双重积分算子,可实现空间token之间的非局部相互作用,其数据依赖型核函数刻画了从数据到底层算子隐藏参数场的逆映射。由此,注意力机制可从多系统生成的训练数据中提取全局先验信息,并以非线性核映射的形式给出探索空间。因此,NAO能够通过编码正则化并实现泛化性,解决PDE反问题中的不适定性与秩亏问题。我们通过实验证明,在对未见过的数据分辨率和系统状态的泛化能力方面,NAO优于基线神经模型。本研究不仅为学习物理系统的可解释基础模型提供了一种新型神经算子架构,也为理解注意力机制提供了新视角。
英文摘要:
Despite the recent popularity of attention-based neural architectures in core AI fields like natural language processing (NLP) and computer vision (CV), their potential in modeling complex physical systems remains under-explored. Learning problems in physical systems are often characterized as discovering operators that map between function spaces based on a few instances of function pairs. This task frequently presents a severely ill-posed PDE inverse problem. In this work, we propose a novel neural operator architecture based on the attention mechanism, which we coin Nonlocal Attention Operator (NAO), and explore its capability towards developing a foundation physical model. In particular, we show that the attention mechanism is equivalent to a double integral operator that enables nonlocal interactions among spatial tokens, with a data-dependent kernel characterizing the inverse mapping from data to the hidden parameter field of the underlying operator. As such, the attention mechanism extracts global prior information from training data generated by multiple systems, and suggests the exploratory space in the form of a nonlinear kernel map. Consequently, NAO can address ill-posedness and rank deficiency in inverse PDE problems by encoding regularization and achieving generalizability. We empirically demonstrate the advantages of NAO over baseline neural models in terms of generalizability to unseen data resolutions and system states. Our work not only suggests a novel neural operator architecture for learning interpretable foundation models of physical systems, but also offers a new perspective towards understanding the attention mechanism.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。