$\ell_p$ 敏感度采样的更紧界限
Sharper Bounds for $\ell_p$ Sensitivity Sampling
AI总结:
本文针对 $p>2$ 的 $\ell_p$ 子空间嵌入提出超越 $\mathfrak S d$ 的一般界限的敏感度采样新边界,并改进了根杠杆得分等采样算法的界限,为结构化矩阵提供了最佳样本复杂度。
摘要:
在大规模机器学习中,随机采样是一种通过少量具有代表性的样本子集来近似数据集的流行方法。特别是,敏感度采样是一种被深入研究的技术,它提供了关于近似质量的可证明保证,同时在极其广泛的设置下将样本数量减少到 VC 维 $d$ 和总敏感度 $\mathfrak S$ 的乘积。然而,尽管先前对敏感度采样进行了深入研究,超越 $\mathfrak S d$ 这一一般界限的保证可能仅在某一种设置(即 $\ell_2$ 子空间嵌入)中已知。在本研究中,我们针对 $p > 2$ 的 $\ell_p$ 子空间嵌入的敏感度采样给出了首个超越一般 $\mathfrak S d$ 界限的边界,对于 $2<p<\infty$ 实现了大约为 $\mathfrak S^{2-2/p}$ 的界限。此外,我们的技术在采样算法研究中产生了新的结果,表明根杠杆得分采样算法对于 $1\leq p<2$ 实现了大约为 $d$ 的界限,并且杠杆得分与敏感度采样的组合对于 $2<p<\infty$ 实现了大约为 $d^{2/p}\mathfrak S^{2-4/p}$ 的改进界限。我们的敏感度采样结果为一大类具有小 $\ell_p$ 敏感度的结构化矩阵提供了已知最佳的样本复杂度。
英文摘要:
In large scale machine learning, random sampling is a popular way to approximate datasets by a small representative subset of examples. In particular, sensitivity sampling is an intensely studied technique which provides provable guarantees on the quality of approximation, while reducing the number of examples to the product of the VC dimension $d$ and the total sensitivity $\mathfrak S$ in remarkably general settings. However, guarantees going beyond this general bound of $\mathfrak S d$ are known in perhaps only one setting, for $\ell_2$ subspace embeddings, despite intense study of sensitivity sampling in prior work. In this work, we show the first bounds for sensitivity sampling for $\ell_p$ subspace embeddings for $p > 2$ that improve over the general $\mathfrak S d$ bound, achieving a bound of roughly $\mathfrak S^{2-2/p}$ for $2<p<\infty$. Furthermore, our techniques yield further new results in the study of sampling algorithms, showing that the root leverage score sampling algorithm achieves a bound of roughly $d$ for $1\leq p<2$, and that a combination of leverage score and sensitivity sampling achieves an improved bound of roughly $d^{2/p}\mathfrak S^{2-4/p}$ for $2<p<\infty$. Our sensitivity sampling results yield the best known sample complexity for a wide class of structured matrices that have small $\ell_p$ sensitivity.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。