多级训练用于柯莫罗戈夫-阿诺德网络
Multilevel Training for Kolmogorov Arnold Networks
AI总结：
本文提出多级训练方法，利用KANs的结构特性提升训练效率，尤其在物理信息神经网络中表现显著。
摘要：
算法加速常见神经网络架构的训练过程受到其固有函数组合所保证的结构缺乏的阻碍。与多层感知机（MLPs）不同，柯莫罗戈夫-阿诺德网络（KANs）通过在指定基底中扩展学习激活函数提供更多的结构。本文利用这种结构，开发了实用算法和理论见解，通过多级训练实现KANs的训练加速。为此，我们首先建立了具有样条基函数的KANs与具有幂ReLU激活函数的多通道MLP之间的等价性，通过线性基变换。然后分析这种基变换如何影响梯度基于优化相对于样条结点的几何结构。KANs的基变换激励了多级训练方法，其中我们训练一系列自然定义通过均匀细化样条结点的KANs，通过模型之间的解析几何插值算子。插值方案使架构形成“适当嵌套的层次结构”，确保插值到精细模型时保留粗模型上的进展，同时样条基函数的紧凑支持确保后续层次的互补优化。数值实验表明，我们的多级训练方法可以实现传统方法在训练可比KANs或MLP时的准确性提升数量级，特别是在物理信息神经网络中。最后，这项工作展示了神经网络原理化设计如何导致可利用的结构，并在此情况下，多级算法可以大幅提高训练性能。
英文摘要：
Algorithmic speedup of training common neural architectures is made difficult by the lack of structure guaranteed by the function compositions inherent to such networks. In contrast to multilayer perceptrons (MLPs), Kolmogorov-Arnold networks (KANs) provide more structure by expanding learned activations in a specified basis. This paper exploits this structure to develop practical algorithms and theoretical insights, yielding training speedup via multilevel training for KANs. To do so, we first establish an equivalence between KANs with spline basis functions and multichannel MLPs with power ReLU activations through a linear change of basis. We then analyze how this change of basis affects the geometry of gradient-based optimization with respect to spline knots. The KANs change-of-basis motivates a multilevel training approach, where we train a sequence of KANs naturally defined through a uniform refinement of spline knots with analytic geometric interpolation operators between models. The interpolation scheme enables a ``properly nested hierarchy'' of architectures, ensuring that interpolation to a fine model preserves the progress made on coarse models, while the compact support of spline basis functions ensures complementary optimization on subsequent levels. Numerical experiments demonstrate that our multilevel training approach can achieve orders of magnitude improvement in accuracy over conventional methods to train comparable KANs or MLPs, particularly for physics informed neural networks. Finally, this work demonstrates how principled design of neural networks can lead to exploitable structure, and in this case, multilevel algorithms that can dramatically improve training performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
