AF-KAN：基于激活函数的Kolmogorov-Arnold网络用于高效表示学习
AF-KAN: Activation Function-Based Kolmogorov-Arnold Networks for Efficient Representation Learning
AI总结：
本文提出AF-KAN，通过多种激活函数组合和注意力机制扩展ReLU-KAN，在相同参数量下显著优于MLP及其他KAN，并探索了最优配置。
摘要：
Kolmogorov-Arnold网络（KANs）已激发大量工作，探索其在广泛科学问题中的应用，并有望替代多层感知机（MLPs）。虽然许多KAN使用基函数和多项式函数（如B样条）设计，但ReLU-KAN利用ReLU函数的组合来模拟B样条结构并发挥ReLU的速度优势。然而，ReLU-KAN并非为多输入构建，其局限性源于ReLU对负值的处理，这可能限制特征提取。为解决这些问题，我们提出基于激活函数的Kolmogorov-Arnold网络（AF-KAN），通过多种激活函数及其函数组合扩展ReLU-KAN。这种新型KAN还引入参数缩减方法，主要是注意力机制和数据归一化，以提升在图像分类数据集上的性能。我们探索了不同的激活函数、函数组合、网格大小和样条阶数，以验证AF-KAN的有效性并确定其最优配置。在实验中，AF-KAN在相同参数数量下显著优于MLP、ReLU-KAN和其他KAN。即使在参数数量少6到10倍的情况下，保持相同网络结构，它仍具有竞争力。然而，AF-KAN需要更长的训练时间并消耗更多FLOPs。本工作的代码库可在https://github.com/hoangthangta/All-KAN获取。
英文摘要：
Kolmogorov-Arnold Networks (KANs) have inspired numerous works exploring their applications across a wide range of scientific problems, with the potential to replace Multilayer Perceptrons (MLPs). While many KANs are designed using basis and polynomial functions, such as B-splines, ReLU-KAN utilizes a combination of ReLU functions to mimic the structure of B-splines and take advantage of ReLU's speed. However, ReLU-KAN is not built for multiple inputs, and its limitations stem from ReLU's handling of negative values, which can restrict feature extraction. To address these issues, we introduce Activation Function-Based Kolmogorov-Arnold Networks (AF-KAN), expanding ReLU-KAN with various activations and their function combinations. This novel KAN also incorporates parameter reduction methods, primarily attention mechanisms and data normalization, to enhance performance on image classification datasets. We explore different activation functions, function combinations, grid sizes, and spline orders to validate the effectiveness of AF-KAN and determine its optimal configuration. In the experiments, AF-KAN significantly outperforms MLP, ReLU-KAN, and other KANs with the same parameter count. It also remains competitive even when using fewer than 6 to 10 times the parameters while maintaining the same network structure. However, AF-KAN requires a longer training time and consumes more FLOPs. The repository for this work is available at https://github.com/hoangthangta/All-KAN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
