发表机构
Nanyang Technological University; CFAR, A*STAR（南洋理工大学; 新加坡科技研究局CFAR）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对PDE基础模型适配需密集解数据的问题，提出无监督微调框架，引入NSLoRA解决LoRA的物理量学习不均问题，在未见PDE上取得优异性能。
AI 中文摘要
预训练的偏微分方程（PDE）基础模型可跨不同方程泛化，但将其适配到未见PDE系统通常需要密集的解数据，这类数据往往昂贵或不可用。为解决这一局限，我们提出一种基于PDE的无监督微调框架，无需真实解。我们首先在涵盖不同空间尺度的各类时变PDE上预训练邻域注意力Transformer，得到可跨异构方程迁移的表示。在适配阶段，我们利用PDE残差和边界条件构建基于物理的目标函数，通过低秩适配（LoRA）对模型在未见方程上进行微调。为解决标准LoRA中物理量学习不均的问题，我们引入NSLoRA，这是一种牛顿-舒尔茨正交化变体，可重新平衡适配。我们的方法在无需任何真实解的情况下，取得了与有监督LoRA微调相当的性能，且在涵盖多个空间维度的异构PDE基准测试中，始终优于竞争型神经算子基线和最新PDE基础模型。
英文摘要
Pretrained partial differential equation (PDE) foundation models can generalize across different equations, but adapting them to unseen PDE systems typically requires dense solution data, which is often expensive or unavailable. To address this limitation, we propose an unsupervised PDE-based finetuning framework that eliminates the need for ground-truth solutions. We first pretrain a neighborhood attention Transformer on diverse time-dependent PDEs spanning varying spatial scales, yielding transferable representations across heterogeneous equations. In the adaptation stage, we construct a physics-based objective using the PDE residual and boundary conditions, and finetune the model on unseen equations via low-rank adaptation (LoRA). To address the uneven learning across physical quantities in standard LoRA, we introduce NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation. Our method achieves performance comparable to supervised LoRA finetuning without requiring any ground-truth solutions, while consistently outperforming competitive neural operator baselines and recent PDE foundation models across heterogeneous PDE benchmarks spanning multiple spatial dimensions.
