arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.14882cs.LGcs.AIcs.NAmath.NA
收藏

Kolmogorov-Arnold网络在含噪函数上有效性的降低

Reduced Effectiveness of Kolmogorov-Arnold Networks on Functions with Noise

Southeast University(东南大学);School of Information Science and Engineering, Southeast University(东南大学信息科学与工程学院);School of Economics and Management, Southeast University(东南大学经济管理学院)

Haoran Shen, Chen Zeng, Jiahui Wang, Qiao Wang

AI总结:

本研究定量评估噪声对KAN性能的影响,提出过采样与扩散映射核滤波相结合的方法,发现测试损失随数据量增加按$r^{-1/2}$下降,但噪声仍削弱其有效性。

摘要:

已有观察表明,即使数据集中引入少量噪声,也会显著降低KAN的性能。在本简短报告中,我们旨在定量评估向数据集添加噪声时的性能表现。我们提出了一种过采样与去噪相结合的技术,以减轻噪声的影响。具体而言,我们采用基于扩散映射的核滤波对含噪数据进行预过滤,以训练KAN网络。我们的实验表明,在添加具有任意固定信噪比的独立同分布噪声时,当训练数据量增加$r$倍,KAN的测试损失(RMSE)将呈现出如$\text{test-loss} \sim \mathcal{O}(r^{-\frac{1}{2}})$(当$r\to +\infty$时)的性能趋势。我们得出结论,同时应用过采样和滤波策略可以减少噪声的有害影响。然而,确定核滤波过程的最优方差具有挑战性,且增加训练数据量会大幅增加相关成本,因为训练数据集需要相对于初始干净数据扩展多倍。因此,数据中的噪声最终削弱了Kolmogorov-Arnold网络的有效性。

英文摘要:

It has been observed that even a small amount of noise introduced into the dataset can significantly degrade the performance of KAN. In this brief note, we aim to quantitatively evaluate the performance when noise is added to the dataset. We propose an oversampling technique combined with denoising to alleviate the impact of noise. Specifically, we employ kernel filtering based on diffusion maps for pre-filtering the noisy data for training KAN network. Our experiments show that while adding i.i.d. noise with any fixed SNR, when we increase the amount of training data by a factor of $r$, the test-loss (RMSE) of KANs will exhibit a performance trend like $\text{test-loss} \sim \mathcal{O}(r^{-\frac{1}{2}})$ as $r\to +\infty$. We conclude that applying both oversampling and filtering strategies can reduce the detrimental effects of noise. Nevertheless, determining the optimal variance for the kernel filtering process is challenging, and enhancing the volume of training data substantially increases the associated costs, because the training dataset needs to be expanded multiple times in comparison to the initial clean data. As a result, the noise present in the data ultimately diminishes the effectiveness of Kolmogorov-Arnold networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑