核求积的谱界
Spectral Bounds for Kernel Quadrature
浏览论文内容
中文总结 AI 辅助
本文针对机器学习核方法存储瓶颈问题，提出用求积公式离散积分得到的核近似原核特征值，在高斯核等案例中验证其近似效果优于蒙特卡洛方法。
中文摘要 AI 辅助
机器学习中核方法理论的一个瓶颈是存储需求。为缓解这一问题，常用技巧是用显式特征映射替代核函数。最著名的例子是高斯核，它可通过傅里叶特征表示。解析上，核函数$K$可表示为包含特征映射对应的可能非对称核$G$的积分表达式。数值上，需用合适的数值积分方案（通常是蒙特卡洛）近似该积分。本文证明，用合适求积公式离散积分得到的核的特征值，对$K$的特征值的近似效果远优于蒙特卡洛近似。我们在高斯核、对应于sigmoid和ReLU激活函数的四维欧氏空间单位球上的神经正切核的案例中验证了这一结论。
英文摘要
A bottleneck in the theory of kernel methods in machine learning is the storage requirement. To ameliorate this, a standard trick is to replace the kernel with an explicit feature map. Perhaps, the most well known example is the Gaussian kernel which can be expressed in terms of the Fourier features. Analytically, the kernel $K$ can be expressed in terms of an integral expression that involves a possibly asymmetric kernel $G$ representing the feature map. Numerically, one needs to approximate this integral by a suitable numerical integration scheme, typically Monte Carlo. In this paper, we demonstrate that the eigenvalues of $K$ are approximated much better by the eigenvalues of the kernel obtained by discretizing the integral using suitable quadrature formulas instead. We illustrate this fact in the case of the Gaussian kernel and neural tangent kernels on the unit sphere of a four-dimensional Euclidean space corresponding to the sigmoid and ReLU activation functions.
发表机构
- University of California, San Diego（加州大学圣地亚哥分校）
- University of New South Wales（新南威尔士大学）
- Claremont Graduate University（克莱蒙特研究生大学）
机构由 AI 辅助整理，请以论文原文为准。
