k-分区上的统计数据的散列
Hashing for statistics over k-partitions
摘要:
在本文中,我们分析了一个哈希函数$k$ -分区成箱集,获得强浓度边界的标准算法相结合的统计数据,从每个箱。这种通用方法最初是由Flajolet和Martin~[FOCS'83]提出的,目的是在估计数据流中不同元素的数量时,在$k$独立样本上节省每个元素的时间因子$Ω(k)$。Flajolet等人的HyperLogLog算法也使用了它。[AOFA'97]和Li等人的大规模机器学习。[NIPS[12]对集合相似性的最小估计。$k$ -分区的主要问题是,当使用流行的散列函数时,不同箱的内容可能高度相关。这意味着分析单个箱的边际分布的方法不适用。在这里,我们表明,一个基于列表的哈希函数,混合列表,并产生强大的浓度界限上最流行的应用程序的$k$ -分区类似于那些我们会得到使用一个真正的随机哈希函数。分析是非常复杂的,并意味着几个新的结果的独立利益,为简单和双重制表,例如一个简单而有效的建设可逆布隆过滤器和均匀散列在一个给定的集合。
英文摘要:
In this paper we analyze a hash function for $k$-partitioning a set into bins, obtaining strong concentration bounds for standard algorithms combining statistics from each bin. This generic method was originally introduced by Flajolet and Martin~[FOCS'83] in order to save a factor $Ω(k)$ of time per element over $k$ independent samples when estimating the number of distinct elements in a data stream. It was also used in the widely used HyperLogLog algorithm of Flajolet et al.~[AOFA'97] and in large-scale machine learning by Li et al.~[NIPS'12] for minwise estimation of set similarity. The main issue of $k$-partition, is that the contents of different bins may be highly correlated when using popular hash functions. This means that methods of analyzing the marginal distribution for a single bin do not apply. Here we show that a tabulation based hash function, mixed tabulation, does yield strong concentration bounds on the most popular applications of $k$-partitioning similar to those we would get using a truly random hash function. The analysis is very involved and implies several new results of independent interest for both simple and double tabulation, e.g. a simple and efficient construction for invertible bloom filters and uniform hashing on a given set.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。