簇注意力神经算子用于求解参数化偏微分方程
Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations
- University of Chinese Academy of Sciences（中国科学院大学）
- Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）
- International School for Advanced Studies (SISSA)（国际高等研究学院（SISSA））
- Zhongyuan University of Technology（中原工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出簇注意力神经算子CANO，通过动态聚类查询并保留全分辨率键值的交叉注意力机制，在保持计算效率的同时提升参数化PDE求解精度，在多个基准上达到最先进性能。
AI中文摘要：
参数化偏微分方程（PDE）的传统模拟依赖于对每个参数进行重复计算，这使得高保真设计变得不切实际。神经算子通过学习解算子来解决这一问题，将参数空间映射的速度提升了多个数量级。近年来，基于Transformer的神经算子试图捕获全局依赖关系，但往往以二次方注意力复杂度为代价。Transolver通过将物理状态投影到缩减的切片空间中进行注意力计算来解决此问题。尽管速度很快，但这种投影牺牲了精细的空间信息。此外，在这种缩减空间中使用跨注意力头共享权重进行操作，可能限制模型的灵活性，从而限制其捕获复杂现象的能力。为了解决这些问题，我们提出了簇注意力神经算子（CANO），它通过一种新颖的交叉注意力机制重新构建了注意力，该机制动态地聚类查询，同时保留全分辨率的键和值。这避免了切片压缩损失，并消除了权重共享的限制。同时，该模型在保持快速的同时不丢失全局交互。实验上，CANO在标准PDE基准上取得了最先进的性能，涵盖了流体和固体动力学（例如，Navier-Stokes、Airfoil、Plasticity）、不规则的非结构化几何（例如，Pipe Turbulence、Composites）以及长期时间滚动。在固体变形和湍流基准上，CANO实现了比基线更低的误差，并表现出强大的几何适应性和时间一致性。
英文摘要：
Traditional simulations of parametric partial differential equations (PDEs) rely on repetitive computations for each parameter, which makes high-fidelity design impractical. Neural operators address this issue by learning solution operators, accelerating parameter-space mapping by orders of magnitude. Recent Transformer-based neural operators attempt to capture global dependencies, but often at the cost of quadratic attention complexity. Transolver resolves this problem by projecting physical states into a reduced slice space for attention computation. Although fast, this projection sacrifices fine spatial information. Moreover, by operating in this reduced space with shared weights across attention heads, it may constrain the model's flexibility, thereby limiting its capacity to capture complex phenomena. To address these issues, we propose the Cluster Attention Neural Operator (CANO), which reformulates attention via a novel cross-attention mechanism that dynamically clusters queries while preserving full-resolution keys and values. This avoids slice compression loss and removes weight-sharing limits. At the same time, the model remains fast without losing global interactions. Empirically, CANO achieves state-of-the-art performance across canonical PDE benchmarks, covering fluid and solid dynamics (e.g., Navier-Stokes, Airfoil, Plasticity), irregular unstructured geometries (e.g., Pipe Turbulence, Composites), and long-term temporal rollouts. Across solid deformation and turbulent flow benchmarks, CANO achieves lower errors than baselines and exhibits strong geometric adaptability and temporal consistency.
