基于自适应采样的近输入稀疏时间核嵌入
Near Input Sparsity Time Kernel Embeddings via Adaptive Sampling
摘要:
为了加速核方法,我们提出了一个近输入稀疏时间算法采样的高维特征空间隐式定义的核变换。我们的主要贡献是一种重要性采样方法,用于在几乎输入稀疏时间内对数据点的度$q$张量的特征空间进行子采样,改进了最近的不经意草图绘制方法(Ahle等人,2020年)的系数为$q^{5/2}/ε^2$。这导致多项式核以及高斯核的子空间嵌入,其中目标维度仅线性依赖于核的统计维度,并且在时间上仅线性依赖于输入数据集的稀疏性。我们展示了我们的子空间嵌入边界如何暗示核岭回归的新统计保证。此外,我们的经验表明,在大规模回归任务中,我们的算法优于最先进的核近似方法。
英文摘要:
To accelerate kernel methods, we propose a near input sparsity time algorithm for sampling the high-dimensional feature space implicitly defined by a kernel transformation. Our main contribution is an importance sampling method for subsampling the feature space of a degree $q$ tensoring of data points in almost input sparsity time, improving the recent oblivious sketching method of (Ahle et al., 2020) by a factor of $q^{5/2}/ε^2$. This leads to a subspace embedding for the polynomial kernel, as well as the Gaussian kernel, with a target dimension that is only linearly dependent on the statistical dimension of the kernel and in time which is only linearly dependent on the sparsity of the input dataset. We show how our subspace embedding bounds imply new statistical guarantees for kernel ridge regression. Furthermore, we empirically show that in large-scale regression tasks, our algorithm outperforms state-of-the-art kernel approximation methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。