arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.17616cs.LG
收藏

在低维度上预训练神经算子

Pretraining a Neural Operator in Lower Dimensions

Carnegie Mellon University(卡内基梅隆大学)

AmirPouya Hemmasian, Amir Barati Farimani

AI总结:

本文提出 PreLowD 策略,在数据成本最低的低维 PDE 上预训练分解傅里叶神经算子,并研究微调配置以提升其在高维相似 PDE 上的求解效果。

摘要:

近年来,通过大规模预训练开发基础神经偏微分方程(PDE)求解器和神经算子受到越来越多的关注。然而,与利用丰富且廉价(无标签)数据进行预训练的视觉和语言模型不同,这些神经求解器通常依赖模拟的 PDE 数据,而获取这些数据可能代价高昂,尤其是对于高维 PDE。在这项工作中,我们旨在在数据收集成本最低的低维 PDE 上预训练神经 PDE 求解器(PreLowD)。我们评估了这种预训练策略在更高维度的相似 PDE 上的有效性。我们使用分解傅里叶神经算子(FFNO),因为它具有必要的灵活性,可应用于任意空间维度的 PDE 数据,并能在低维度中重用已训练的参数。此外,我们的工作揭示了微调配置的影响,以充分利用这种预训练策略。代码可在 https://github.com/BaratiLab/PreLowD 获取。

英文摘要:

There has recently been increasing attention towards developing foundational neural Partial Differential Equation (PDE) solvers and neural operators through large-scale pretraining. However, unlike vision and language models that make use of abundant and inexpensive (unlabeled) data for pretraining, these neural solvers usually rely on simulated PDE data, which can be costly to obtain, especially for high-dimensional PDEs. In this work, we aim to Pretrain neural PDE solvers on Lower Dimensional PDEs (PreLowD) where data collection is the least expensive. We evaluated the effectiveness of this pretraining strategy in similar PDEs in higher dimensions. We use the Factorized Fourier Neural Operator (FFNO) due to having the necessary flexibility to be applied to PDE data of arbitrary spatial dimensions and reuse trained parameters in lower dimensions. In addition, our work sheds light on the effect of the fine-tuning configuration to make the most of this pretraining strategy. Code is available at https://github.com/BaratiLab/PreLowD.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑