arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.01433cs.LGstat.ML
收藏

较小核心集的紧灵敏度界限

Tight Sensitivity Bounds For Smaller Coresets

University of Haifa(海法大学)

Alaa Maalouf, Adiel Statman, Dan Feldman

摘要:

$\varepsilon$ -矩阵$A\in{\mathbb{R}}^{n\times d}$的最小均方(LMS)的核心集是其行的一个小加权子集,该子集近似于从其行到${\mathbb{R}}^d$的每个仿射$k$维子空间的平方距离之和,最多为$1\pm\varepsilon$的一个因子。这样的核心集对于超参数调整和解决许多最小均方问题是有用的,例如低秩近似($k$ -SVD)、$k$ -PCA、Lassso/Ridge/Linear regression等等。Coreset也适用于并行处理流式、动态和分布式大数据。在高概率的情况下,基于$A$中每行的重要性或敏感性的上限的非均匀采样产生coreset。(采样)coreset的大小,然后在这些灵敏度界限的总和接近线性。我们提供的算法,计算可证明的\emph{tight}边界的敏感性,每个输入行。它是基于两个成分:(i)迭代算法,计算精确的灵敏度,每个点到任意小精度(非仿射)$k$ -子空间,和(ii)一般减少的独立利益,从计算灵敏度为家庭的仿射$k$ -子空间在${\mathbb{R}}^d$(非仿射)$(k+1)$ -子空间在${\mathbb{R}}^{d+1}$。真实世界的数据集上的实验结果,包括英文维基百科文档术语矩阵,表明我们的界限提供显着更小的和数据依赖的coresets也在实践中。还提供了完全的开源。

英文摘要:

An $\varepsilon$-coreset for Least-Mean-Squares (LMS) of a matrix $A\in{\mathbb{R}}^{n\times d}$ is a small weighted subset of its rows that approximates the sum of squared distances from its rows to every affine $k$-dimensional subspace of ${\mathbb{R}}^d$, up to a factor of $1\pm\varepsilon$. Such coresets are useful for hyper-parameter tuning and solving many least-mean-squares problems such as low-rank approximation ($k$-SVD), $k$-PCA, Lassso/Ridge/Linear regression and many more. Coresets are also useful for handling streaming, dynamic and distributed big data in parallel. With high probability, non-uniform sampling based on upper bounds on what is known as importance or sensitivity of each row in $A$ yields a coreset. The size of the (sampled) coreset is then near-linear in the total sum of these sensitivity bounds. We provide algorithms that compute provably \emph{tight} bounds for the sensitivity of each input row. It is based on two ingredients: (i) iterative algorithm that computes the exact sensitivity of each point up to arbitrary small precision for (non-affine) $k$-subspaces, and (ii) a general reduction of independent interest from computing sensitivity for the family of affine $k$-subspaces in ${\mathbb{R}}^d$ to (non-affine) $(k+1)$- subspaces in ${\mathbb{R}}^{d+1}$. Experimental results on real-world datasets, including the English Wikipedia documents-term matrix, show that our bounds provide significantly smaller and data-dependent coresets also in practice. Full open source is also provided.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑