Reduced Effectiveness of Kolmogorov-Arnold Networks on Functions with Noise
Southeast University(东南大学);School of Information Science and Engineering, Southeast University(东南大学信息科学与工程学院);School of Economics and Management, Southeast University(东南大学经济管理学院)
It has been observed that even a small amount of noise introduced into the dataset can significantly degrade the performance of KAN. In this brief note, we aim to quantitatively evaluate the performance when noise is added to the dataset. We propose an oversampling technique combined with denoising to alleviate the impact of noise. Specifically, we employ kernel filtering based on diffusion maps for pre-filtering the noisy data for training KAN network. Our experiments show that while adding i.i.d. noise with any fixed SNR, when we increase the amount of training data by a factor of $r$, the test-loss (RMSE) of KANs will exhibit a performance trend like $\text{test-loss} \sim \mathcal{O}(r^{-\frac{1}{2}})$ as $r\to +\infty$. We conclude that applying both oversampling and filtering strategies can reduce the detrimental effects of noise. Nevertheless, determining the optimal variance for the kernel filtering process is challenging, and enhancing the volume of training data substantially increases the associated costs, because the training dataset needs to be expanded multiple times in comparison to the initial clean data. As a result, the noise present in the data ultimately diminishes the effectiveness of Kolmogorov-Arnold networks.
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Tsinghua University(清华大学);Bauhaus-Universität Weimar(魏玛包豪斯大学);CNPC Engineering Technology RD Company Limited(中国石油集团工程技术研究院有限公司);Queensland University of Technology(昆士兰科技大学);Leibniz University Hannover(汉诺威莱布尼茨大学)
A Practitioner's Guide to Kolmogorov-Arnold Networks
Department of Mathematics, Hong Kong Baptist University(香港 Baptist大学数学系);Institution for Foundations of Data Science, Yale University(数据科学基础研究所,耶鲁大学);Robotics and Technology of Computers Lab., Universidad de Sevilla(机器人与计算机技术实验室,塞维利亚大学)
Kolmogorov-Arnold networks for metal surface defect classification
AGH University of Krakow(克拉科夫AGH大学);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所);University of Technology Sydney(悉尼科技大学);Óbuda University(欧布达大学);NASK National Research Institute(NASK国家研究院)
Sinc Kolmogorov-Arnold network and its application for solving PDEs with singularities
Skolkovo Institute of Science and Technology(斯克洛夫科学与技术研究所);Southern University of Science and Technology(南方科技大学);International Center for Mathematics(国际数学中心);National Center for Applied Mathematics Shenzhen (NCAMS)(深圳应用数学中心)
HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation
the Faculty of Electrical Engineering, Czestochowa University of Technology(捷斯托霍瓦技术大学电子工程系);the Faculty of Mathematics and Computer Science, University of Lodz(罗兹大学数学与计算机科学系);the Centre for Data Analysis, Modelling and Computational Sciences (CAMINO), University of Lodz(罗兹大学数据分析、建模与计算科学中心)
Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation
Indian Institute of Science Education and Research Mohali(印度科学教育与研究研究所莫哈利分校);Indian Institute of Science Education and Research Thiruvananthapuram(印度科学教育与研究研究所蒂鲁文南特布勒姆分校)
Department of Computer, Control and Management Engineering, Sapienza University of Rome(西帕辛扎大学罗马分校计算机、控制与管理工程系);Department of Life Science Informatics and Data Science, B-IT, LIMES Program Unit Chemical Biology and Medicinal Chemistry, Rheinische Friedrich-Wilhelms-Universität, Bonn, Germany(波恩德国莱茵-威斯巴登大学生命科学信息技术与数据科学系);Lamarr Institute for Machine Learning and Artificial Intelligence, Bonn, Germany(波恩德国拉马尔机器学习与人工智能研究所)