CountSketches，特征散列和三个的中值
CountSketches, Feature Hashing and the Median of Three
摘要：
在本文中，我们重新审视经典的CountSketch方法，这是一个稀疏的，随机投影，转换（高维）欧几里得向量$v$到一个向量的维度$(2t-1) s$，其中$t, s > 0$是整数参数。已知即使对于$t=1$，CountSketch也允许估计$v$的坐标，其中方差由$\|v\|_2^2/s$限定。对于$t > 1$，估计量取$2t-1$独立估计值的中位数，并且估计值偏离$2 \|v\|_2/\sqrt{s}$以上的概率在$t$中呈指数级小。这表明选择$t$为所需逆失效概率的对数。然而，CountSketch的实现通常使用一个小的、恒定的$t$。以前的工作只预测在这种情况下，一个恒定的因素改善。我们的主要贡献是一个新的分析计数草图，显示了改善方差$O(\min\{\|v\|_1^2/s^2,\|v\|_2^2/s\})$时，$t > 1$。也就是说，对于足够大的$s$，方差与$s^{-2}$成比例地渐近减小。我们还研究了方差的设置中的内积是从两个CountSketches估计。这一发现表明，特征散列方法本质上与CountSketch相同，但不使用中值估计器，可以在可能使用中值估计器的设置中以较小的成本变得更可靠。我们在实验中证实了我们的理论发现，从而有助于证明为什么在实践中，一个小的常数的估计往往是足够的。我们改进的方差界是基于新的一般定理的方差和高的时刻的中位数的独立同分布。随机变量，可能是独立的利益。
英文摘要：
In this paper, we revisit the classic CountSketch method, which is a sparse, random projection that transforms a (high-dimensional) Euclidean vector $v$ to a vector of dimension $(2t-1) s$, where $t, s > 0$ are integer parameters. It is known that even for $t=1$, a CountSketch allows estimating coordinates of $v$ with variance bounded by $\|v\|_2^2/s$. For $t > 1$, the estimator takes the median of $2t-1$ independent estimates, and the probability that the estimate is off by more than $2 \|v\|_2/\sqrt{s}$ is exponentially small in $t$. This suggests choosing $t$ to be logarithmic in a desired inverse failure probability. However, implementations of CountSketch often use a small, constant $t$. Previous work only predicts a constant factor improvement in this setting. Our main contribution is a new analysis of Count-Sketch, showing an improvement in variance to $O(\min\{\|v\|_1^2/s^2,\|v\|_2^2/s\})$ when $t > 1$. That is, the variance decreases proportionally to $s^{-2}$, asymptotically for large enough $s$. We also study the variance in the setting where an inner product is to be estimated from two CountSketches. This finding suggests that the Feature Hashing method, which is essentially identical to CountSketch but does not make use of the median estimator, can be made more reliable at a small cost in settings where using a median estimator is possible. We confirm our theoretical findings in experiments and thereby help justify why a small constant number of estimates often suffice in practice. Our improved variance bounds are based on new general theorems about the variance and higher moments of the median of i.i.d. random variables that may be of independent interest.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
