arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.07187cs.LG
收藏

UPS:通过跨模态适配高效构建用于 PDE 求解的基础模型

UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptation

Carnegie Mellon University(卡内基梅隆大学)

Junhong Shen, Tanya Marwah, Ameet Talwalkar

AI总结:

本文提出统一 PDE 求解器 UPS,通过将不同 PDE 嵌入共享表示空间并结合 FNO-transformer 架构与预训练 LLM 热启动和显式对齐,在 PDEBench 上以更少数据和计算量实现最先进性能,并支持少样本迁移。

摘要:

我们提出统一 PDE 求解器(UPS),这是一种数据和计算高效的方法,用于为来自不同领域、维度和分辨率的多种时空 PDE 族开发统一神经算子。UPS 将不同的 PDE 嵌入到一个共享表示空间中,并使用 FNO-transformer 架构对其进行处理。我们没有从头开始训练网络,因为这种方式数据需求大且计算成本高,而是从预训练的 LLM 热启动 transformer,并进行显式对齐以减少模态差距,同时提高数据和计算效率。跨模态 UPS 在 PDEBench 中的广泛 1D 和 2D PDE 族上取得了最先进的结果,以少 4 倍的数据和少 26 倍的计算量超越了现有的统一模型。同时,它能够对未见过的 PDE 族和系数进行少样本迁移。

英文摘要:

We present Unified PDE Solvers (UPS), a data- and compute-efficient approach to developing unified neural operators for diverse families of spatiotemporal PDEs from various domains, dimensions, and resolutions. UPS embeds different PDEs into a shared representation space and processes them using a FNO-transformer architecture. Rather than training the network from scratch, which is data-demanding and computationally expensive, we warm-start the transformer from pretrained LLMs and perform explicit alignment to reduce the modality gap while improving data and compute efficiency. The cross-modal UPS achieves state-of-the-art results on a wide range of 1D and 2D PDE families from PDEBench, outperforming existing unified models using 4 times less data and 26 times less compute. Meanwhile, it is capable of few-shot transfer to unseen PDE families and coefficients.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑