QuadNorm:用于神经算子的分辨率鲁棒归一化
QuadNorm: Resolution-Robust Normalization for Neural Operators
AI总结:
本文提出QuadNorm和BlendQuadNorm,通过数值求积替代传统均匀平均,提升神经算子在不同分辨率下的鲁棒性,实验显示其在非周期性PDE和非谱架构中表现最佳。
摘要:
神经算子中的归一化层通常通过均匀平均离散网格值计算统计信息,导致归一化过程依赖于离散化,从而在不同分辨率或网格间产生转移误差。为实现离散化鲁棒性,我们引入了四则归一化家族,用数值求积替代归一化层中的均匀平均:QuadNorm和BlendQuadNorm。在端点包含的均匀网格上,所提出的四则矩在不同离散化中具有O(h²)一致性,意味着其跨分辨率不匹配随网格间距平方衰减。一个转移误差界则预测归一化引起的不匹配如何随分辨率差距和网络深度变化。实验显示与转移误差界预测的相同差距和深度缩放趋势。在Darcy问题中,QuadNorm在64²到256²的每个测试目标分辨率上表现最佳;在真实数据基准测试中,Transolver配合QuadNorm实现几乎分辨率不变的转移。最大的收益出现在非周期性PDE和非谱架构中,其中原分辨率改进也显现。我们还验证了BlendQuadNorm,其接近LayerNorm行为,并在周期性FNO设置中作为保守的默认选项。这些结果表明归一化是神经算子中此前被忽视的分辨率依赖源。
英文摘要:
Normalization layers in neural operators usually compute statistics by uniformly averaging discrete grid values, making the normalization itself discretization-dependent and thereby a source of transfer error across different resolutions or meshes. To enable discretization robustness, we introduce a quadrature normalization family that replaces existing uniform averaging in normalization layers with numerical quadrature: QuadNorm and BlendQuadNorm. On endpoint-inclusive uniform grids, the proposed quadrature moments are $O(h^2)$-consistent across discretizations, meaning that their cross-resolution mismatch decays quadratically with grid spacing. A transfer-error bound then predicts how normalization-induced mismatch scales with both the resolution gap and network depth. The experiments show the same gap- and depth-scaling trends predicted by the transfer-error bound. On Darcy, QuadNorm delivers the best cross-resolution performance at every tested target resolution from $64^2$ to $256^2$; on real-data benchmarks, Transolver with QuadNorm achieves nearly resolution-invariant transfer. The largest gains appear on nonperiodic PDEs and nonspectral architectures, where native-resolution improvements also emerge. We also validate BlendQuadNorm, which stays close to LayerNorm behavior and serves as a conservative default for periodic FNO settings. These results identify normalization as a previously overlooked source of resolution dependence in neural operators.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。