arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.01156cs.DS
收藏

无重复:具有高度集中散列的快速流

No Repetition: Fast Streaming with Highly Concentrated Hashing

Anders Aamand, Debarati Das, Evangelos Kipouridis, Jakob B. T. Knudsen, Peter M. R. Rasmussen, Mikkel Thorup

摘要:

为了获得在一定误差范围内以高概率工作的估计器,一种常见的策略是设计一种以恒定概率工作的估计器,然后使用独立重复来提高概率。这种方法的重要例子是小空间算法,用于估计流中不同元素的数量,或估计大集合之间的集合相似性。使用标准的强通用散列来处理每个元素,我们得到了一个基于草图的估计器,其中太大错误的概率是1/4。通过执行$r$独立重复并取估计量的中值,错误概率在$r$中呈指数下降。但是,运行$r$独立实验会使处理时间增加$r$倍。在这里,我们指出,如果我们有一个具有强浓度边界的哈希函数,那么我们可以得到相同的高概率边界,而无需重复。与$r$独立的草图不同,我们有一个比$r$大一倍的草图,因此总空间是相同的。然而,我们只应用了一个哈希函数,所以我们及时地节省了一个因子$r$,整个算法变得更简单了。具有强浓度边界的快速实用哈希函数最近由Aamand em等人提出(将出现在STOC 2020中)。使用他们的散列方案,算法因此变得非常快速和实用,适合于在线处理大容量的数据流。

英文摘要:

To get estimators that work within a certain error bound with high probability, a common strategy is to design one that works with constant probability, and then boost the probability using independent repetitions. Important examples of this approach are small space algorithms for estimating the number of distinct elements in a stream, or estimating the set similarity between large sets. Using standard strongly universal hashing to process each element, we get a sketch based estimator where the probability of a too large error is, say, 1/4. By performing $r$ independent repetitions and taking the median of the estimators, the error probability falls exponentially in $r$. However, running $r$ independent experiments increases the processing time by a factor $r$. Here we make the point that if we have a hash function with strong concentration bounds, then we get the same high probability bounds without any need for repetitions. Instead of $r$ independent sketches, we have a single sketch that is $r$ times bigger, so the total space is the same. However, we only apply a single hash function, so we save a factor $r$ in time, and the overall algorithms just get simpler. Fast practical hash functions with strong concentration bounds were recently proposed by Aamand em et al. (to appear in STOC 2020). Using their hashing schemes, the algorithms thus become very fast and practical, suitable for online processing of high volume data streams.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑