关于Kolmogorov-Arnold网络训练的研究
On Training of Kolmogorov-Arnold Networks
AI总结：
本文研究KAN与MLP的训练动态，发现KAN在高维数据上有效且参数效率更优，但训练不稳定，并给出提升稳定性的建议。
摘要：
Kolmogorov-Arnold网络最近被引入作为多层感知器架构的灵活替代方案。在本文中，我们研究了不同KAN架构的训练动态，并将其与相应的MLP公式进行比较。我们使用多种不同的初始化方案、优化器和学习率进行训练，并利用如HSIC瓶颈等免反向传播的方法。我们发现（以测试准确率衡量）KAN在高维数据集上是MLP架构的有效替代方案，且具有更好的参数效率，但训练动态更不稳定。最后，我们为提高较大KAN模型的训练稳定性提供了建议。
英文摘要：
Kolmogorov-Arnold Networks have recently been introduced as a flexible alternative to multi-layer Perceptron architectures. In this paper, we examine the training dynamics of different KAN architectures and compare them with corresponding MLP formulations. We train with a variety of different initialization schemes, optimizers, and learning rates, as well as utilize back propagation free approaches like the HSIC Bottleneck. We find that (when judged by test accuracy) KANs are an effective alternative to MLP architectures on high-dimensional datasets and have somewhat better parameter efficiency, but suffer from more unstable training dynamics. Finally, we provide recommendations for improving training stability of larger KAN models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
