深度学习中的频率原理/谱偏置概述
Overview frequency principle/spectral bias in deep learning
AI总结：
本文概述了深度学习中的频率原理（F-Principle/谱偏置），指出 DNNs 在训练时倾向于从低频到高频拟合函数，揭示了其学习低频函数的优势和高频函数的不足，并提出了未来研究的开放问题。
摘要：
随着深度学习日益深入工业界和科学领域，理解深度学习变得愈发迫切。近年来，基于傅里叶分析的研究路线通过展示深度神经网络（DNNs）训练行为中的频率原理（F-Principle 或谱偏置），为这一神奇的“黑箱”提供了启示——DNNs 在训练过程中通常从低频到高频逐步拟合函数。F-Principle 首先在一维合成数据上得到验证，随后在高维真实数据集中得到证实。一系列后续工作进一步增强了 F-Principle 的有效性。这种低频隐式偏置揭示了神经网络在学习低频函数上的优势及其在学习高频函数上的不足。这种理解启发了实际问题中基于 DNN 的算法设计，解释了各种场景下出现的实验现象，并进一步推动了从频率视角对深度学习的研究。尽管尚不完整，我们提供了 F-Principle 的概述并提出了一些未来研究的开放问题。
英文摘要：
Understanding deep learning is increasingly emergent as it penetrates more and more into industry and science. In recent years, a research line from Fourier analysis sheds lights on this magical "black box" by showing a Frequency Principle (F-Principle or spectral bias) of the training behavior of deep neural networks (DNNs) -- DNNs often fit functions from low to high frequency during the training. The F-Principle is first demonstrated by onedimensional synthetic data followed by the verification in high-dimensional real datasets. A series of works subsequently enhance the validity of the F-Principle. This low-frequency implicit bias reveals the strength of neural network in learning low-frequency functions as well as its deficiency in learning high-frequency functions. Such understanding inspires the design of DNN-based algorithms in practical problems, explains experimental phenomena emerging in various scenarios, and further advances the study of deep learning from the frequency perspective. Although incomplete, we provide an overview of F-Principle and propose some open problems for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
