本文并未提出新方法,而是针对 KAN 与 MLP 模型在机器学习、计算机视觉、音频处理、自然语言处理、符号公式表示等各类任务上开展了更公平全面的对比。具体而言,我们控制参数量和 FLOPs 来比较二者的性能。主要观察结果为:除符号公式表示任务外,MLP 的表现普遍优于 KAN。我们还对 KAN 开展了消融研究,发现其在符号公式表示任务上的优势主要源于 B 样条激活函数;将 B 样条应用于 MLP 后,其在符号公式表示任务上的性能显著提升,达到或超过 KAN 的水平。但在 MLP 原本就优于 KAN 的其他任务中,B 样条并未大幅提升 MLP 的性能。此外,我们发现在标准的类别增量持续学习设置下,KAN 的遗忘问题比 MLP 更严重,这与 KAN 原论文的结论不同。项目链接:https://github.com/yu-rp/KANbeFair
英文摘要:
This paper does not introduce a novel method. Instead, it offers a fairer and more comprehensive comparison of KAN and MLP models across various tasks, including machine learning, computer vision, audio processing, natural language processing, and symbolic formula representation. Specifically, we control the number of parameters and FLOPs to compare the performance of KAN and MLP. Our main observation is that, except for symbolic formula representation tasks, MLP generally outperforms KAN. We also conduct ablation studies on KAN and find that its advantage in symbolic formula representation mainly stems from its B-spline activation function. When B-spline is applied to MLP, performance in symbolic formula representation significantly improves, surpassing or matching that of KAN. However, in other tasks where MLP already excels over KAN, B-spline does not substantially enhance MLP's performance. Furthermore, we find that KAN's forgetting issue is more severe than that of MLP in a standard class-incremental continual learning setting, which differs from the findings reported in the KAN paper. We hope these results provide insights for future research on KAN and other MLP alternatives. Project link: https://github.com/yu-rp/KANbeFair
Is the Statistical Advantage Worth the Cost? An Empirical Comparison of KANs and MLPs for Structured Data Classification
Institute of Computer Science, University of the Philippines Los Baños(菲律宾大学洛斯巴尼奥斯分校计算机科学研究所);Machine Learning and Artificial Intelligence Applications Lab, University of the Philippines Los Baños(菲律宾大学洛斯巴尼奥斯分校机器学习与人工智能应用实验室)
School of Information Science and Engineering, Southeast University(东南大学信息科学与工程学院);School of Economics and Management, Southeast University(东南大学经济管理学院)
MLPs and KANs for data-driven learning in physical problems: A performance comparison
The Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学学院);Texas Advanced Computing Center (TACC)(德克萨斯高级计算中心);Department of Civil, Architecture and Environmental Engineering(土木、建筑与环境工程系);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
Kolmogorov Arnold networks (KAN) for aerodynamic prediction: a comparison with MLPs and GNNs
ETSIAE-UPM-School of Aeronautics, Universidad Politécnica de Madrid(马德里理工大学航空学院);Center for Computational Simulation, Universidad Politécnica de Madrid(马德里理工大学计算模拟中心);Institute for Cross-Disciplinary Physics and Complex Systems (IFISC, CSIC-UIB)(跨学科物理与复杂系统研究所(IFISC,CSIC-UIB))
School of Advanced Interdisciplinary Sciences, UCAS(中国科学院大学前沿交叉科学学院);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);Institute of Software, CAS(中国科学院软件研究所)
KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search
University of Helsinki(赫尔辛基大学);Polish Academy of Sciences(波兰科学院);Silesian University of Technology(西里西亚工业大学);Advanced Research Centre(高级研究中心);Delft University of Technology(代尔夫特理工大学);Raman Research Institute(拉曼研究所);International Institute of Information Technology, Hyderabad(海得拉巴国际信息技术学院)
Hybrid Parallel Bessel KAN Model: a Kolmogorov Arnold Network: integrated framework for developing a novel architecture to reconstruct the fused-basis approach via a parameter free quadratic interaction
Tezpur University(提斯普尔大学);Pandit Deendayal Energy University(潘迪特·迪恩戴亚尔能源大学)
Multilevel Training for Kolmogorov Arnold Networks
Theoretical Division, Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室理论 division);Center for Computing Research, Sandia National Laboratories(桑迪亚国家实验室计算研究中心)
A Practitioner's Guide to Kolmogorov-Arnold Networks
Department of Mathematics, Hong Kong Baptist University(香港 Baptist大学数学系);Institution for Foundations of Data Science, Yale University(数据科学基础研究所,耶鲁大学);Robotics and Technology of Computers Lab., Universidad de Sevilla(机器人与计算机技术实验室,塞维利亚大学)
Kolmogorov-Arnold Networks for Personal Context Recognition on ExtraSensory
University of Information Technology, Ho Chi Minh City, Vietnam(胡志明市信息技术大学);Vietnam National University Ho Chi Minh City, Ho Chi Minh City, Vietnam(越南国立大学胡志明市分校)
School of Information, Renmin University of China(中国人民大学信息学院);Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, China(教育部数据工程与知识工程重点实验室);School of Economics and Management, Tsinghua University, China(清华大学经济管理学院)
Incorporating Arbitrary Matrix Group Equivariance into KANs
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(人工智能基础理论国家重点实验室,智能科学与技术学院,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China(黄埔实验室(琶洲),广州,广东,中国)
Kolmogorov-Arnold networks for metal surface defect classification
AGH University of Krakow(克拉科夫AGH大学);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所);University of Technology Sydney(悉尼科技大学);Óbuda University(欧布达大学);NASK National Research Institute(NASK国家研究院)