优化、泛化与梯度下降在科莫戈洛夫-阿诺尔德网络中的差分隐私界限
Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networks
AI总结：
本文研究了梯度下降在两层科莫戈洛夫-阿诺尔德网络中的优化、泛化及差分隐私性质，推导出泛化界限，并在逻辑损失下证明了多项对数宽度足以实现优化和泛化速率，同时揭示了差分隐私下宽度的必要性。
摘要：
科莫戈洛夫-阿诺尔德网络（KANs）最近作为标准MLP的结构替代方案出现，但其训练动态、泛化和隐私属性的系统理论仍有限。本文分析了梯度下降（GD）训练两层KANs，并推导出一般界限，以表征其训练动态、泛化和差分隐私下的效用。作为具体实例，我们专注于逻辑损失，在NTK可分离假设下，证明多项对数宽度足以使GD达到优化速率$1/T$和泛化速率$1/n$，其中$T$为GD迭代次数，$n$为样本量。在隐私设置下，我们表征所需的噪声以实现$(ε,δ)$-DP，并获得效用界$\sqrt{d}/(nε)$（$d$为输入维度），与经典凸Lipschitz问题的下界匹配。我们的结果表明，多项对数宽度不仅足够而且在差分隐私下是必要的，揭示了非隐私（仅充分性）和隐私（也出现必要性）训练制度之间的定性差距。实验进一步说明了这些理论见解如何指导实际选择，包括网络宽度选择和早停。
英文摘要：
Kolmogorov--Arnold Networks (KANs) have recently emerged as a structured alternative to standard MLPs, yet a principled theory for their training dynamics, generalization, and privacy properties remains limited. In this paper, we analyze gradient descent (GD) for training two-layer KANs and derive general bounds that characterize their training dynamics, generalization, and utility under differential privacy (DP). As a concrete instantiation, we specialize our analysis to logistic loss under an NTK-separable assumption, where we show that polylogarithmic network width suffices for GD to achieve an optimization rate of order $1/T$ and a generalization rate of order $1/n$, with $T$ denoting the number of GD iterations and $n$ the sample size. In the private setting, we characterize the noise required for $(ε,δ)$-DP and obtain a utility bound of order $\sqrt{d}/(nε)$ (with $d$ the input dimension), matching the classical lower bound for general convex Lipschitz problems. Our results imply that polylogarithmic width is not only sufficient but also necessary under differential privacy, revealing a qualitative gap between non-private (sufficiency only) and private (necessity also emerges) training regimes. Experiments further illustrate how these theoretical insights can guide practical choices, including network width selection and early stopping.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
