各向同性核如何在简单不变量上执行
How isotropic kernels perform on simple invariants
摘要：
我们研究了各向同性核方法的训练曲线如何取决于要学习的任务的对称性，在几种设置。(i)我们考虑一个回归任务，目标函数是一个高斯随机场，它只依赖于$d_\parallel$变量，小于输入维度$d$。我们计算$ε\sim p^{-β}$之后的预期测试误差$ε$，其中$p$是训练集的大小。我们发现，$β\sim 1/d$独立的$d_\parallel$，支持以前的研究结果，即不变量的存在并不能解决核回归的维数灾难。(ii)接下来，我们考虑支持向量二进制分类，并介绍了条纹模型，其中的数据标签依赖于一个单一的坐标$y(\underline{x}) = y(x_1)$，对应于平行的决策边界分离标签的不同符号，并认为在这些接口没有余量。我们认为，并确认数值大带宽，$β= \frac{d-1+ξ}{3d-3+ξ}$，其中$ξ\in (0,2)$是指数的奇异性内核在原点。该估计改进了从Rademacher复杂性可获得的经典界限。在此设置中，由于$β\rightarrow 1 / 3$为$d\rightarrow\infty$，因此不存在维数灾难。(iii)我们证实了这些发现的球形模型，其中$y(\underline{x}) = y(|\underline{x}|)$。(iv)在条带模型中，我们证明了如果数据沿着它们的不变量被某个因子$λ$（一种被认为发生在深度网络中的操作）压缩，则测试误差会减少一个因子$λ^{-\frac{2(d-1)}{3d-3+ξ}}$。
英文摘要：
We investigate how the training curve of isotropic kernel methods depends on the symmetry of the task to be learned, in several settings. (i) We consider a regression task, where the target function is a Gaussian random field that depends only on $d_\parallel$ variables, fewer than the input dimension $d$. We compute the expected test error $ε$ that follows $ε\sim p^{-β}$ where $p$ is the size of the training set. We find that $β\sim 1/d$ independently of $d_\parallel$, supporting previous findings that the presence of invariants does not resolve the curse of dimensionality for kernel regression. (ii) Next we consider support-vector binary classification and introduce the stripe model where the data label depends on a single coordinate $y(\underline{x}) = y(x_1)$, corresponding to parallel decision boundaries separating labels of different signs, and consider that there is no margin at these interfaces. We argue and confirm numerically that for large bandwidth, $β= \frac{d-1+ξ}{3d-3+ξ}$, where $ξ\in (0,2)$ is the exponent characterizing the singularity of the kernel at the origin. This estimation improves classical bounds obtainable from Rademacher complexity. In this setting there is no curse of dimensionality since $β\rightarrow 1 / 3$ as $d\rightarrow\infty$. (iii) We confirm these findings for the spherical model for which $y(\underline{x}) = y(|\underline{x}|)$. (iv) In the stripe model, we show that if the data are compressed along their invariants by some factor $λ$ (an operation believed to take place in deep networks), the test error is reduced by a factor $λ^{-\frac{2(d-1)}{3d-3+ξ}}$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
