Kolmogorov-Arnold Network (KAN) is a network structure recently proposed by Liu et al. (2024) that offers improved interpretability and a more parsimonious design in many science-oriented tasks compared to multi-layer perceptrons. This work provides a rigorous theoretical analysis of KAN by establishing generalization bounds for KAN equipped with activation functions that are either represented by linear combinations of basis functions or lying in a low-rank Reproducing Kernel Hilbert Space (RKHS). In the first case, the generalization bound accommodates various choices of basis functions in forming the activation functions in each layer of KAN and is adapted to different operator norms at each layer. For a particular choice of operator norms, the bound scales with the $l_1$ norm of the coefficient matrices and the Lipschitz constants for the activation functions, and it has no dependence on combinatorial parameters (e.g., number of nodes) outside of logarithmic factors. Moreover, our result does not require the boundedness assumption on the loss function and, hence, is applicable to a general class of regression-type loss functions. In the low-rank case, the generalization bound scales polynomially with the underlying ranks as well as the Lipschitz constants of the activation functions in each layer. These bounds are empirically investigated for KANs trained with stochastic gradient descent on simulated and real data sets. The numerical results demonstrate the practical relevance of these bounds.
University of Twente（特文特大学）；Leiden University（莱顿大学）
There is a longstanding debate whether the Kolmogorov-Arnold representation theorem can explain the use of more than one hidden layer in neural networks. The Kolmogorov-Arnold representation decomposes a multivariate function into an interior and an outer function and therefore has indeed a similar structure as a neural network with two hidden layers. But there are distinctive differences. One of the main obstacles is that the outer function depends on the represented function and can be wildly varying even if the represented function is smooth. We derive modifications of the Kolmogorov-Arnold…
Kolmogorov-Arnold networks for metal surface defect classification
AGH University of Krakow（克拉科夫AGH大学）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）；University of Technology Sydney（悉尼科技大学）；Óbuda University（欧布达大学）；NASK National Research Institute（NASK国家研究院）
Mathematics & Statistics, University of Vermont（数学与统计学系，佛蒙特大学）；Vermont Complex Systems Center, University of Vermont（佛蒙特复杂系统中心，佛蒙特大学）；Computer Science, University of Vermont（计算机科学系，佛蒙特大学）
Columbia University（哥伦比亚大学）；National University of Singapore（新加坡国立大学）
We prove a theorem concerning the approximation of multivariate functions by deep ReLU networks, for which the curse of the dimensionality is lessened. Our theorem is based on a constructive proof of the Kolmogorov--Arnold superposition theorem, and on a subset of multivariate continuous functions whose outer superposition functions can be efficiently approximated by deep ReLU networks.
Optimized Architectures for Kolmogorov-Arnold Networks
Mathematics & Statistics, University of Vermont（佛蒙特大学数学与统计学系）；Vermont Complex Systems Center, University of Vermont（佛蒙特复杂系统中心）；Computer Science, University of Vermont（佛蒙特大学计算机科学系）
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Tsinghua University（清华大学）；Bauhaus-Universität Weimar（魏玛包豪斯大学）；CNPC Engineering Technology RD Company Limited（中国石油集团工程技术研究院有限公司）；Queensland University of Technology（昆士兰科技大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
A Practitioner's Guide to Kolmogorov-Arnold Networks
Department of Mathematics, Hong Kong Baptist University（香港 Baptist大学数学系）；Institution for Foundations of Data Science, Yale University（数据科学基础研究所，耶鲁大学）；Robotics and Technology of Computers Lab., Universidad de Sevilla（机器人与计算机技术实验室，塞维利亚大学）
State-Space Kolmogorov Arnold Networks for Interpretable Nonlinear System Identification
Faculty of Engineering Technology, Vrije Universiteit Brussel（自由大学布鲁塞尔工程科技学院）；Department of Artificial Intelligence and Systems Engineering, Budapest University of Technology and Economics（布达佩斯技术与经济大学人工智能与系统工程系）
Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of North Carolina, Charlotte（北卡罗来纳大学夏洛特分校）；Delft University of Technology（代尔夫特理工大学）；University of Washington, Applied Mathematics（华盛顿大学应用数学系）；Brown University, Applied Mathematics（布朗大学应用数学系）
