本文分析了 Kolmogorov-Arnold 网络(KAN)的普适逼近性质,证明了当所有边缘函数为仿射时普适性不成立,而添加一个非仿射函数即可恢复普适性,并给出了深层和两层 KAN 的充要条件。
摘要:
我们从边缘函数的角度分析了 Kolmogorov-Arnold 网络(KAN)的普适逼近性质。如果这些函数都是仿射的,那么普适性显然不成立。除了仿射函数之外,还需要多少非仿射函数才能保证普适性?我们证明一个就足够了。更精确地说,我们证明对于每个紧集 $K\subset\mathbb{R}^n$,所有边缘函数要么是仿射的,要么等于一个固定的连续函数 $\sigma$ 的深层 KAN 在 $C(K)$ 中稠密,当且仅当 $\sigma$ 是非仿射的。相比之下,对于恰好有两个隐藏层的 KAN,普适性成立当且仅当 $\sigma$ 是非多项式的。我们进一步证明,并不需要完整的仿射函数类;它可以用一个有限集代替而不影响普适性。特别地,在非多项式情况下,当深度任意时,一个由五个仿射函数组成的固定族就足够了。更一般地,对于每个连续的非仿射函数 $\sigma$,存在一个有限的仿射族 $A_\sigma$,使得边缘函数在 $A_\sigma\cup\{\sigma\}$ 中的深层 KAN 仍然是普适的。我们还证明,采用 Liu 等人~\cite{Liu2024} 引入的基于样条的边缘参数化的 KAN 是经典意义上的普适逼近器,即使样条次数和节点序列是预先固定的。
英文摘要:
We analyze the universal approximation property of Kolmogorov-Arnold Networks (KANs) in terms of their edge functions. If these functions are all affine, then universality clearly fails. How many non-affine functions are needed, in addition to affine ones, to ensure universality? We show that a single one suffices. More precisely, we prove that deep KANs in which all edge functions are either affine or equal to a fixed continuous function $σ$ are dense in $C(K)$ for every compact set $K\subset\mathbb{R}^n$ if and only if $σ$ is non-affine. In contrast, for KANs with exactly two hidden layers, universality holds if and only if $σ$ is nonpolynomial. We further show that the full class of affine functions is not required; it can be replaced by a finite set without affecting universality. In particular, in the nonpolynomial case, a fixed family of five affine functions suffices when the depth is arbitrary. More generally, for every continuous non-affine function $σ$, there exists a finite affine family $A_σ$ such that deep KANs with edge functions in $A_σ\cup\{σ\}$ remain universal. We also prove that KANs with the spline-based edge parameterization introduced by Liu et al.~\cite{Liu2024} are universal approximators in the classical sense, even when the spline degree and knot sequence are fixed in advance.
Kolmogorov-Arnold networks for metal surface defect classification
AGH University of Krakow(克拉科夫AGH大学);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所);University of Technology Sydney(悉尼科技大学);Óbuda University(欧布达大学);NASK National Research Institute(NASK国家研究院)
Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks
Tsinghua University(清华大学);Bauhaus-Universität Weimar(魏玛包豪斯大学);CNPC Engineering Technology RD Company Limited(中国石油集团工程技术研究院有限公司);Queensland University of Technology(昆士兰科技大学);Leibniz University Hannover(汉诺威莱布尼茨大学)
A Practitioner's Guide to Kolmogorov-Arnold Networks
Department of Mathematics, Hong Kong Baptist University(香港 Baptist大学数学系);Institution for Foundations of Data Science, Yale University(数据科学基础研究所,耶鲁大学);Robotics and Technology of Computers Lab., Universidad de Sevilla(机器人与计算机技术实验室,塞维利亚大学)
Mathematics & Statistics, University of Vermont(数学与统计学系,佛蒙特大学);Vermont Complex Systems Center, University of Vermont(佛蒙特复杂系统中心,佛蒙特大学);Computer Science, University of Vermont(计算机科学系,佛蒙特大学)
State-Space Kolmogorov Arnold Networks for Interpretable Nonlinear System Identification
Faculty of Engineering Technology, Vrije Universiteit Brussel(自由大学布鲁塞尔工程科技学院);Department of Artificial Intelligence and Systems Engineering, Budapest University of Technology and Economics(布达佩斯技术与经济大学人工智能与系统工程系)
Optimized Architectures for Kolmogorov-Arnold Networks
Mathematics & Statistics, University of Vermont(佛蒙特大学数学与统计学系);Vermont Complex Systems Center, University of Vermont(佛蒙特复杂系统中心);Computer Science, University of Vermont(佛蒙特大学计算机科学系)
Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems
Pacific Northwest National Laboratory(太平洋西北国家实验室);University of North Carolina, Charlotte(北卡罗来纳大学夏洛特分校);Delft University of Technology(代尔夫特理工大学);University of Washington, Applied Mathematics(华盛顿大学应用数学系);Brown University, Applied Mathematics(布朗大学应用数学系)
Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China(电子科技大学基础与前沿科学研究所);Potsdam Institute for Climate Impact Research(波茨坦气候影响研究所);Department of Physics, Humboldt University Berlin(柏林洪堡大学物理系);Research Institute of Intelligent Complex Systems, Fudan University(复旦大学智能复杂系统研究所);School of Physics, University of Electronic Science and Technology of China(电子科技大学物理学院);Key Laboratory of Quantum Physics and Photonic Quantum Information, Ministry of Education, University of Electronic Science and Technology of China(电子科技大学教育部量子物理与光子量子信息重点实验室);Non-classical Information Science Basic Discipline Research Center of Sichuan Province, University of Electronic Science and Technology of China(电子科技大学四川省非经典信息科学基础学科研究中心)
HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation
the Faculty of Electrical Engineering, Czestochowa University of Technology(捷斯托霍瓦技术大学电子工程系);the Faculty of Mathematics and Computer Science, University of Lodz(罗兹大学数学与计算机科学系);the Centre for Data Analysis, Modelling and Computational Sciences (CAMINO), University of Lodz(罗兹大学数据分析、建模与计算科学中心)