arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.00435cs.LG
收藏

将任意矩阵群等变性融入KANs

Incorporating Arbitrary Matrix Group Equivariance into KANs

State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(人工智能基础理论国家重点实验室,智能科学与技术学院,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);Pazhou Laboratory (Huangpu), Guangzhou, Guangdong, China(黄埔实验室(琶洲),广州,广东,中国)

Lexiang Hu, Yisen Wang, Zhouchen Lin

AI总结:

提出等变Kolmogorov-Arnold Networks (EKAN),通过构建门控样条基函数与等变线性权重融入任意矩阵群等变性,在对称性相关任务上以更少参数和数据实现更高精度。

摘要:

得益于样条激活函数,Kolmogorov-Arnold Networks (KANs) 在科学领域取得了巨大成功,成为 Multi-Layer Perceptrons (MLPs) 的替代方案。然而,样条函数可能无法尊重任务中的对称性,而这是机器学习中关键的先验知识。在本文中,我们提出了 Equivariant Kolmogorov-Arnold Networks (EKAN),一种将任意矩阵群等变性融入 KANs 的方法,旨在拓宽其在更多领域的适用性。我们首先构建门控样条基函数,它们与等变线性权重共同构成 EKAN 层,然后定义一个提升层以将 EKAN 的输入空间与数据集的特征空间对齐,从而构建整个 EKAN 架构。与基线模型相比,EKAN 在对称性相关任务(如粒子散射和三体问题)中,以更小的数据集或更少的参数实现了更高的准确率,通常将测试 MSE 降低几个数量级。即使在非符号公式场景中,例如具有三个喷流成分的 top quark 标记,EKAN 也使用不到 40% 的参数取得了与最先进等变架构相当的结果,而 KANs 并未如预期般优于 MLPs。代码和数据可在 https://github.com/hulx2002/EKAN 获取。

英文摘要:

Kolmogorov-Arnold Networks (KANs) have seen great success in scientific domains thanks to spline activation functions, becoming an alternative to Multi-Layer Perceptrons (MLPs). However, spline functions may not respect symmetry in tasks, which is crucial prior knowledge in machine learning. In this paper, we propose Equivariant Kolmogorov-Arnold Networks (EKAN), a method for incorporating arbitrary matrix group equivariance into KANs, aiming to broaden their applicability to more fields. We first construct gated spline basis functions, which form the EKAN layer together with equivariant linear weights, and then define a lift layer to align the input space of EKAN with the feature space of the dataset, thereby building the entire EKAN architecture. Compared with baseline models, EKAN achieves higher accuracy with smaller datasets or fewer parameters on symmetry-related tasks, such as particle scattering and the three-body problem, often reducing test MSE by several orders of magnitude. Even in non-symbolic formula scenarios, such as top quark tagging with three jet constituents, EKAN achieves comparable results with state-of-the-art equivariant architectures using fewer than 40% of the parameters, while KANs do not outperform MLPs as expected. Code and data are available at https://github.com/hulx2002/EKAN .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑