用神经网络求解偏微分方程时损失切换中的损失跳变
Loss Jump During Loss Switch in Solving PDEs with Neural Networks
AI总结：
本文研究神经网络求解PDE时不同损失函数的影响，发现从数据损失切换到模型损失会引发稳定的损失跳变现象，并通过实验和理论分析揭示其源于不同损失函数下神经网络的频率偏好差异。
摘要：
使用神经网络求解偏微分方程（PDEs）正在科学计算领域作为一种替代方法日益流行。神经网络可以将不同类型的信息整合到损失函数中，包括观测数据、控制方程和变分形式等。这些损失函数可大致分为两类：观测数据损失直接约束并衡量模型输出，而其他损失函数间接建模网络性能，可归类为模型损失。然而，这种替代方法对其底层机制仍缺乏透彻理解，包括理论基础和对各种现象的严格刻画。本工作重点研究不同损失函数如何影响用于求解PDEs的神经网络训练。我们发现了一个稳定的损失跳变现象：当损失函数从数据损失切换到包含不同阶导数信息的模型损失时，神经网络解会立即显著偏离精确解。进一步实验表明，该现象源于神经网络在不同损失函数下具有不同的频率偏好。我们从理论上分析了模型损失下神经网络的频率偏好。这一损失跳变现象为审视神经网络求解PDEs的底层机制提供了有价值的视角。
英文摘要：
Using neural networks to solve partial differential equations (PDEs) is gaining popularity as an alternative approach in the scientific computing community. Neural networks can integrate different types of information into the loss function. These include observation data, governing equations, and variational forms, etc. These loss functions can be broadly categorized into two types: observation data loss directly constrains and measures the model output, while other loss functions indirectly model the performance of the network, which can be classified as model loss. However, this alternative approach lacks a thorough understanding of its underlying mechanisms, including theoretical foundations and rigorous characterization of various phenomena. This work focuses on investigating how different loss functions impact the training of neural networks for solving PDEs. We discover a stable loss-jump phenomenon: when switching the loss function from the data loss to the model loss, which includes different orders of derivative information, the neural network solution significantly deviates from the exact solution immediately. Further experiments reveal that this phenomenon arises from the different frequency preferences of neural networks under different loss functions. We theoretically analyze the frequency preference of neural networks under model loss. This loss-jump phenomenon provides a valuable perspective for examining the underlying mechanisms of neural networks in solving PDEs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
