Incorporating Arbitrary Matrix Group Equivariance into KANs
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(人工智能基础理论国家重点实验室,智能科学与技术学院,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China(黄埔实验室(琶洲),广州,广东,中国)
得益于样条激活函数,Kolmogorov-Arnold Networks (KANs) 在科学领域取得了巨大成功,成为 Multi-Layer Perceptrons (MLPs) 的替代方案。然而,样条函数可能无法尊重任务中的对称性,而这是机器学习中关键的先验知识。在本文中,我们提出了 Equivariant Kolmogorov-Arnold Networks (EKAN),一种将任意矩阵群等变性融入 KANs 的方法,旨在拓宽其在更多领域的适用性。我们首先构建门控样条基函数,它们与等变线性权重共同构成 EKAN 层,然后定义一个提升层以将 EKAN 的输入空间与数据集的特征空间对齐,从而构建整个 EKAN 架构。与基线模型相比,EKAN 在对称性相关任务(如粒子散射和三体问题)中,以更小的数据集或更少的参数实现了更高的准确率,通常将测试 MSE 降低几个数量级。即使在非符号公式场景中,例如具有三个喷流成分的 top quark 标记,EKAN 也使用不到 40% 的参数取得了与最先进等变架构相当的结果,而 KANs 并未如预期般优于 MLPs。代码和数据可在 https://github.com/hulx2002/EKAN 获取。
英文摘要:
Kolmogorov-Arnold Networks (KANs) have seen great success in scientific domains thanks to spline activation functions, becoming an alternative to Multi-Layer Perceptrons (MLPs). However, spline functions may not respect symmetry in tasks, which is crucial prior knowledge in machine learning. In this paper, we propose Equivariant Kolmogorov-Arnold Networks (EKAN), a method for incorporating arbitrary matrix group equivariance into KANs, aiming to broaden their applicability to more fields. We first construct gated spline basis functions, which form the EKAN layer together with equivariant linear weights, and then define a lift layer to align the input space of EKAN with the feature space of the dataset, thereby building the entire EKAN architecture. Compared with baseline models, EKAN achieves higher accuracy with smaller datasets or fewer parameters on symmetry-related tasks, such as particle scattering and the three-body problem, often reducing test MSE by several orders of magnitude. Even in non-symbolic formula scenarios, such as top quark tagging with three jet constituents, EKAN achieves comparable results with state-of-the-art equivariant architectures using fewer than 40% of the parameters, while KANs do not outperform MLPs as expected. Code and data are available at https://github.com/hulx2002/EKAN .
Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation
Indian Institute of Science Education and Research Mohali(印度科学教育与研究研究所莫哈利分校);Indian Institute of Science Education and Research Thiruvananthapuram(印度科学教育与研究研究所蒂鲁文南特布勒姆分校)
University of Oxford(牛津大学);ELLIS Institute(ELLIS研究所);Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);Tübingen AI Center(图宾根人工智能中心);AITHYRA
Massachusetts Institute of Technology(麻省理工学院);The NSF Institute for Artificial Intelligence and Fundamental Interactions(NSF人工智能与基本相互作用研究所);California Institute of Technology(加州理工学院)
Institute of Mathematics(数学研究所);Image Computing, University of Luebeck, Luebeck, Germany(图像计算,吕贝克大学,吕贝克,德国);Department of Applied Mathematics(应用数学系);Theoretical Physics, University of Cambridge, Cambridge, United Kingdom(理论物理,剑桥大学,剑桥,英国)
HyQuRP: Hybrid quantum-classical neural network with rotational and permutational equivariance
School of Integrated Technology, Yonsei University, Seoul 03722, Korea(整合技术学院,延世大学,首尔03722,韩国);Department of Quantum Information, Yonsei University, Seoul 03722, Korea(量子信息系,延世大学,首尔03722,韩国)
BEKAN: Boundary condition-guaranteed evolutionary Kolmogorov-Arnold networks with radial basis functions for solving PDE problems
School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA(机械工程学院,普渡大学);Department of Mathematics, Purdue University, West Lafayette, IN 47907, USA(数学系,普渡大学)
asKAN: Active Subspace embedded Kolmogorov-Arnold Network
Institute of Mechanics, Chinese Academy of Sciences(中国科学院力学研究所);School of Engineering Sciences, University of Chinese Academy of Sciences(中国科学院大学工程科学学院)
SurvKAN: A Fully Parametric Survival Model Based on Kolmogorov-Arnold Networks
Department of Electronics, Information, and Bioengineering, Politecnico di Milano(电子信息生物工程系,米兰理工学院);Department of Radiation Oncology, Mayo Clinic(放射肿瘤科,梅奥诊所);Division of Radiation Oncology, IEO, European Institute of Oncology, IRCCS(放射肿瘤科,欧洲肿瘤研究所,IRCCS);Department of Oncology and Hemato-Oncology, University of Milan(肿瘤学与血液肿瘤学系,米兰大学)
Evaluation of Sampling Strategies and Physics-Informed Kolmogorov--Arnold Networks in Unbounded Domains
Mathematical Applications in Science and Engineering, School of Applied Sciences and Engineering, Universidad EAFIT, Medellín, Colombia(科学与工程数学应用,应用科学与工程学院,埃法伊特大学,梅德林,哥伦比亚);Nature and City, School of Applied Sciences and Engineering, Universidad EAFIT, Medellín, Colombia(自然与城市,应用科学与工程学院,埃法伊特大学,梅德林,哥伦比亚)