Kolmogorov Arnold Networks上的免事后校准
PostHoc FREE Calibrating on Kolmogorov Arnold Networks
AI总结：
本文研究了KAN的置信度校准问题，分析了四个关键超参数的影响，并提出温度缩放损失(TSL)直接整合温度参数至训练目标，显著降低校准误差并提升预测可靠性。
摘要：
Kolmogorov Arnold Networks (KANs) 是受Kolmogorov Arnold表示定理启发的神经架构，利用B样条参数化进行灵活、局部自适应的函数逼近。尽管KAN能捕捉标准多层感知机(MLP)无法建模的复杂非线性，但其常表现出置信度校准偏差，在密集数据区过度自信，在稀疏区则自信不足。本研究系统考察了层宽、网格阶数、快捷函数和网格范围四个关键超参数对KAN校准的影响。此外，我们提出一种新颖的温度缩放损失(TSL)，将温度参数直接整合进训练目标，在学习过程中动态调整预测分布。标准基准上的理论分析与大量实证评估表明，TSL显著降低了校准误差，提升了概率预测的可靠性。总之，本研究为样条神经网络的设计提供了可操作的见解，并确立TSL作为增强校准的鲁棒损失方案。
英文摘要：
Kolmogorov Arnold Networks (KANs) are neural architectures inspired by the Kolmogorov Arnold representation theorem that leverage B Spline parameterizations for flexible, locally adaptive function approximation. Although KANs can capture complex nonlinearities beyond those modeled by standard MultiLayer Perceptrons (MLPs), they frequently exhibit miscalibrated confidence estimates manifesting as overconfidence in dense data regions and underconfidence in sparse areas. In this work, we systematically examine the impact of four critical hyperparameters including Layer Width, Grid Order, Shortcut Function, and Grid Range on the calibration of KANs. Furthermore, we introduce a novel TemperatureScaled Loss (TSL) that integrates a temperature parameter directly into the training objective, dynamically adjusting the predictive distribution during learning. Both theoretical analysis and extensive empirical evaluations on standard benchmarks demonstrate that TSL significantly reduces calibration errors, thereby improving the reliability of probabilistic predictions. Overall, our study provides actionable insights into the design of spline based neural networks and establishes TSL as a robust loss solution for enhancing calibration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
