arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.03345cs.LG
收藏

Kolmogorov-Arnold网络的硬件导向推理复杂度

Hardware-Oriented Inference Complexity of Kolmogorov-Arnold Networks

Aston Institute of Photonic Technologies(阿斯顿光子技术研究所);Aston University(阿斯顿大学)

Bilal Khalid, Pedro Freire, Sergei K. Turitsyn, Jaroslaw E. Prilepsky

AI总结:

针对KAN在延迟敏感和功耗受限场景下的硬件推理,提出通用平台无关的复杂度度量(RM、BOP、NABS),并扩展到多种KAN变体,实现与其它架构的公平比较。

摘要:

Kolmogorov-Arnold网络(KAN)最近作为一种强大的架构出现在各种机器学习应用中。然而,其独特的结构引发了对其计算开销的显著担忧。现有研究主要根据GPU训练和推理所需的浮点运算(FLOP)来评估KAN复杂度。然而,在许多延迟敏感和功耗受限的部署场景中,例如光通信中神经网络驱动的非线性缓解或无线通信中的信道状态估计,训练是离线进行的,并且专用硬件加速器比GPU更受青睐用于推理。最近的硬件实现研究使用特定平台的资源消耗指标(如查找表、触发器和块RAM)报告KAN复杂度。然而,这些指标需要完整的硬件设计和综合阶段,限制了它们在早期架构决策和跨平台比较中的实用性。为了解决这个问题,我们推导了通用的、平台无关的公式,用于根据实数乘法(RM)、位操作(BOP)以及加法和位移次数(NABS)评估KAN的硬件推理复杂度。我们将分析扩展到多种KAN变体,包括B样条、高斯径向基函数(GRBF)、切比雪夫和傅里叶KAN。所提出的指标可以直接从网络结构计算,并实现KAN与其他神经网络架构之间的公平且直接的推理复杂度比较。

英文摘要:

Kolmogorov-Arnold Networks (KANs) have recently emerged as a powerful architecture for various machine learning applications. However, their unique structure raises significant concerns regarding their computational overhead. Existing studies primarily evaluate KAN complexity in terms of Floating-Point Operations (FLOPs) required for GPU-based training and inference. However, in many latency-sensitive and power-constrained deployment scenarios, such as neural network-driven non-linearity mitigation in optical communications or channel state estimation in wireless communications, training is performed offline and dedicated hardware accelerators are preferred over GPUs for inference. Recent hardware implementation studies report KAN complexity using platform-specific resource consumption metrics, such as Look-Up Tables, Flip-Flops, and Block RAMs. However, these metrics require a full hardware design and synthesis stage that limits their utility for early-stage architectural decisions and cross-platform comparisons. To address this, we derive generalized, platform-independent formulae for evaluating the hardware inference complexity of KANs in terms of Real Multiplications (RM), Bit Operations (BOP), and Number of Additions and Bit-Shifts (NABS). We extend our analysis across multiple KAN variants, including B-spline, Gaussian Radial Basis Function (GRBF), Chebyshev, and Fourier KANs. The proposed metrics can be computed directly from the network structure and enable a fair and straightforward inference complexity comparison between KAN and other neural network architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑