arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.06232cs.DScond-mat.stat-mechcs.CR
收藏

基于NAE-SAT的概率隶属度滤波器

NAE-SAT-based probabilistic membership filters

Chao Fang, Zheng Zhu, Helmut G. Katzgraber

摘要:

概率成员筛选器是一种数据结构,旨在快速验证大型数据集的元素是否属于数据的子集。虽然假阴性是不可能的,但假阳性是可能的。因此,任何好的概率隶属度过滤器的主要目标都是具有较小的误报率,同时具有内存效率和查询速度。虽然Bloom过滤器的构造速度很快,但它们的内存效率受到严格的理论上限的限制。Weaver等人引入了基于随机可满足性的过滤器,显著提高了概率过滤器的效率,然而,在构建过滤器时,代价是求解复杂的随机可满足性(SAT)公式。在这里,我们提出了一个改进的SAT过滤器的方法,重点是减少过滤器的建设时间,以及查询时间。我们的方法是基于使用不全相等(NAE)SAT公式来构建过滤器,通过使用传统的快速随机SAT求解器映射到随机SAT来解决这些问题,以及位打包和减少哈希函数的数量。NAE-SAT过滤器与快速硬件搭配使用,可以产生企业级应用程序。

英文摘要:

Probabilistic membership filters are a type of data structure designed to quickly verify whether an element of a large data set belongs to a subset of the data. While false negatives are not possible, false positives are. Therefore, the main goal of any good probabilistic membership filter is to have a small false-positive rate while being memory efficient and fast to query. Although Bloom filters are fast to construct, their memory efficiency is bounded by a strict theoretical upper bound. Weaver et al. introduced random satisfiability-based filters that significantly improved the efficiency of the probabilistic filters, however, at the cost of solving a complex random satisfiability (SAT) formula when constructing the filter. Here we present an improved SAT filter approach with a focus on reducing the filter building times, as well as query times. Our approach is based on using not-all-equal (NAE) SAT formulas to build the filters, solving these via a mapping to random SAT using traditionally-fast random SAT solvers, as well as bit packing and the reduction of the number of hash functions. Paired with fast hardware, NAE-SAT filters could result in enterprise-size applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑