Kolmogorov-Arnold卷积：设计原则与实证研究
Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies
AI总结：
本文研究KAN在计算机视觉中的卷积化应用，提出参数高效的KA卷积层、微调算法及注意力/焦点调制变体，并通过分类与分割实验验证性能、给出设计指南。
摘要：
Kolmogorov-Arnold Networks（KAN）的出现引发了科学界的广泛关注与争论。本文探索KAN在计算机视觉（CV）领域的应用。我们考察了KAN的卷积版本，考虑样条之外的多种非线性选择，例如小波变换和一系列多项式。我们提出了Kolmogorov-Arnold卷积层的参数高效设计，以及面向预训练KAN模型的参数高效微调算法，还有自注意力层和焦点调制层的KAN卷积版本。我们在用于图像分类任务的MNIST、CIFAR10、CIFAR100、Tiny ImageNet、ImageNet1k和HAM10000数据集上进行了实证评估。此外，我们探索了分割任务，提出了采用KAN卷积的类U-Net架构，并在BUSI、GlaS和CVC数据集上取得了最先进结果。我们将所有发现总结为一份面向计算机视觉任务的KAN卷积模型初步设计指南。此外，我们研究了KAN的正则化技术。所有实验代码以及卷积层和模型的实现（含在ImageNet1k权重上预训练的版本）均可通过GitHub获取：https://github.com/IvanDrokin/torch-conv-kan
英文摘要：
The emergence of Kolmogorov-Arnold Networks (KANs) has sparked significant interest and debate within the scientific community. This paper explores the application of KANs in the domain of computer vision (CV). We examine the convolutional version of KANs, considering various nonlinearity options beyond splines, such as Wavelet transforms and a range of polynomials. We propose a parameter-efficient design for Kolmogorov-Arnold convolutional layers and a parameter-efficient finetuning algorithm for pre-trained KAN models, as well as KAN convolutional versions of self-attention and focal modulation layers. We provide empirical evaluations conducted on MNIST, CIFAR10, CIFAR100, Tiny ImageNet, ImageNet1k, and HAM10000 datasets for image classification tasks. Additionally, we explore segmentation tasks, proposing U-Net-like architectures with KAN convolutions, and achieving state-of-the-art results on BUSI, GlaS, and CVC datasets. We summarized all of our findings in a preliminary design guide of KAN convolutional models for computer vision tasks. Furthermore, we investigate regularization techniques for KANs. All experimental code and implementations of convolutional layers and models, pre-trained on ImageNet1k weights are available on GitHub via this https://github.com/IvanDrokin/torch-conv-kan
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
