arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.02052cs.LG
收藏

鲁棒傅里叶神经网络

Robust Fourier Neural Networks

Halyun Jeong, Jihun Han

AI总结:

本文提出在傅里叶嵌入层后引入对角层以提升网络对测量噪声的鲁棒性,促使网络学习稀疏傅里叶特征,并给出理论依据与数值实验验证。

摘要:

傅里叶嵌入在消除神经网络训练过程中的谱偏置方面展现出巨大潜力。然而,它仍可能遭受较高的泛化误差,尤其是在标签或测量值含有噪声时。我们证明,在傅里叶嵌入层之后引入一个简单的对角层,可以使网络对测量噪声更加鲁棒,从而有效地促使其学习稀疏傅里叶特征。我们利用对角网络和神经网络隐式正则化的最新进展,为这种傅里叶特征学习提供了理论依据。在一定条件下,我们提出的方法还可以学习由傅里叶特征的非线性函数构成的含噪混合函数。数值实验验证了我们所提出架构的有效性,并支持了我们的理论。

英文摘要:

Fourier embedding has shown great promise in removing spectral bias during neural network training. However, it can still suffer from high generalization errors, especially when the labels or measurements are noisy. We demonstrate that introducing a simple diagonal layer after the Fourier embedding layer makes the network more robust to measurement noise, effectively prompting it to learn sparse Fourier features. We provide theoretical justifications for this Fourier feature learning, leveraging recent developments in diagonal networks and implicit regularization in neural networks. Under certain conditions, our proposed approach can also learn functions that are noisy mixtures of nonlinear functions of Fourier features. Numerical experiments validate the effectiveness of our proposed architecture, supporting our theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑