arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.03154cs.DS
收藏

快速和可扩展的最小完美散列,用于大规模密钥集

Fast and scalable minimal perfect hashing for massive key sets

Antoine Limasset, Guillaume Rizk, Rayan Chikhi, Pierre Peterlongo

摘要:

最小完美散列函数在静态集合上提供空间有效和无冲突的散列。由于高构建时间、RAM或外部存储器使用,构建此类函数的现有算法和实现在它们可以处理的输入元素的数量上具有实际限制。我们重新审视了一个简单的算法,并表明它是非常有竞争力的最先进的,特别是在建设时间和内存使用方面。我们提供了一个名为BBhash的并行C++实现。它能够使用8个线程和5 GB内存在不到7分钟的时间内创建$10^{10}$元素的最小完美散列函数,生成的函数使用3.7位/元素。据我们所知,这也是第一个在基数$10^{12}$输入上成功测试的实现。源代码:https://github.com/rizkg/BBHash

英文摘要:

Minimal perfect hash functions provide space-efficient and collision-free hashing on static sets. Existing algorithms and implementations that build such functions have practical limitations on the number of input elements they can process, due to high construction time, RAM or external memory usage. We revisit a simple algorithm and show that it is highly competitive with the state of the art, especially in terms of construction time and memory usage. We provide a parallel C++ implementation called BBhash. It is capable of creating a minimal perfect hash function of $10^{10}$ elements in less than 7 minutes using 8 threads and 5 GB of memory, and the resulting function uses 3.7 bits/element. To the best of our knowledge, this is also the first implementation that has been successfully tested on an input of cardinality $10^{12}$. Source code: https://github.com/rizkg/BBHash

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑