The need for scalable and expressive models in machine learning is paramount, particularly in applications requiring both structural depth and flexibility. Traditional deep learning methods, such as multilayer perceptrons (MLP), offer depth but lack ability to integrate structural characteristics of deep learning architectures with non-parametric flexibility of kernel methods. To address this, deep kernel learning (DKL) was introduced, where inputs to a base kernel are transformed using a deep learning architecture. These kernels can replace standard kernels, allowing both expressive power and scalability. The advent of Kolmogorov-Arnold Networks (KAN) has generated considerable attention and discussion among researchers in scientific domain. In this paper, we introduce a scalable deep kernel using KAN (DKL-KAN) as an effective alternative to DKL using MLP (DKL-MLP). Our approach involves simultaneously optimizing these kernel attributes using marginal likelihood within a Gaussian process framework. We analyze two variants of DKL-KAN for a fair comparison with DKL-MLP: one with same number of neurons and layers as DKL-MLP, and another with approximately same number of trainable parameters. To handle large datasets, we use kernel interpolation for scalable structured Gaussian processes (KISS-GP) for low-dimensional inputs and KISS-GP with product kernels for high-dimensional inputs. The efficacy of DKL-KAN is evaluated in terms of computational training time and test prediction accuracy across a wide range of applications. Additionally, the effectiveness of DKL-KAN is also examined in modeling discontinuities and accurately estimating prediction uncertainty. The results indicate that DKL-KAN outperforms DKL-MLP on datasets with a low number of observations. Conversely, DKL-MLP exhibits better scalability and higher test prediction accuracy on datasets with large number of observations.
Can KAN CANs? Input-convex Kolmogorov-Arnold Networks (KANs) as hyperelastic constitutive artificial neural networks (CANs)
Department of Materials Science and Engineering, Delft University of Technology(材料科学与工程系,代尔夫特理工大学);Department of Biomechanical Engineering, Delft University of Technology(生物机械工程系,代尔夫特理工大学)
The Thørväld Group(Thørväld集团);Inria Paris-Rocquencourt(Inria巴黎-罗克昆court分部);Rajiv Gandhi University Doimukh(拉贾戈夫德大学多伊穆克);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒尔研究实验室);The Kumquat Consortium(Kumquat consortium)
Hybrid Parallel Bessel KAN Model: a Kolmogorov Arnold Network: integrated framework for developing a novel architecture to reconstruct the fused-basis approach via a parameter free quadratic interaction
Tezpur University(提斯普尔大学);Pandit Deendayal Energy University(潘迪特·迪恩戴亚尔能源大学)
Kolmogorov-Arnold Networks for Personal Context Recognition on ExtraSensory
University of Information Technology, Ho Chi Minh City, Vietnam(胡志明市信息技术大学);Vietnam National University Ho Chi Minh City, Ho Chi Minh City, Vietnam(越南国立大学胡志明市分校)
Predictive Modeling of Flexible EHD Pumps using Kolmogorov-Arnold Networks
College of Mechanical Engineering, Chongqing University of Technology(重庆理工大学机械工程学院);Nagoya University(名古屋大学);Faculty of Engineering, Yamaguchi University(山口大学工程学院)
Kolmogorov Arnold networks (KAN) for aerodynamic prediction: a comparison with MLPs and GNNs
ETSIAE-UPM-School of Aeronautics, Universidad Politécnica de Madrid(马德里理工大学航空学院);Center for Computational Simulation, Universidad Politécnica de Madrid(马德里理工大学计算模拟中心);Institute for Cross-Disciplinary Physics and Complex Systems (IFISC, CSIC-UIB)(跨学科物理与复杂系统研究所(IFISC,CSIC-UIB))
Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spaces
Department of Statistics, Seoul National University(首尔国立大学统计系);Department of Statistics, Inha University(inha大学统计系);Department of Mathematics, University of Maryland(马里兰大学数学系)
Kolmogorov-Arnold networks for metal surface defect classification
AGH University of Krakow(克拉科夫AGH大学);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所);University of Technology Sydney(悉尼科技大学);Óbuda University(欧布达大学);NASK National Research Institute(NASK国家研究院)
KANGURA: Kolmogorov-Arnold Network-Based Geometry-Aware Learning with Unified Representation Attention for 3D Modeling of Complex Structures
Department of Electrical Engineering, Iran University of Science and Technology(伊朗科学技术大学电气工程系);Department of Computer Engineering, Iran University of Science and Technology(伊朗科学技术大学计算机工程系);Department of Industrial & Management Systems Engineering, West Virginia University(西弗吉尼亚大学工业与管理系统工程系);H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology(佐治亚理工学院H.米尔顿·斯图尔特工业与系统工程学院);Department of Computer Science, University of California, Davis(加州大学戴维斯分校计算机科学系)
A Practitioner's Guide to Kolmogorov-Arnold Networks
Department of Mathematics, Hong Kong Baptist University(香港 Baptist大学数学系);Institution for Foundations of Data Science, Yale University(数据科学基础研究所,耶鲁大学);Robotics and Technology of Computers Lab., Universidad de Sevilla(机器人与计算机技术实验室,塞维利亚大学)
Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation
Indian Institute of Science Education and Research Mohali(印度科学教育与研究研究所莫哈利分校);Indian Institute of Science Education and Research Thiruvananthapuram(印度科学教育与研究研究所蒂鲁文南特布勒姆分校)
Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementation
Nara Institute of Science and Technology(奈良科学技术研究所);University of Information Technology - VNUHCM(胡志明市国家大学信息学院);Le Quy Don technical University(黎贵敦工业大学)