arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 0910.3349cs.DScs.DBcs.IR
收藏

B-Bit Minwise Hash

b-Bit Minwise Hashing

Ping Li, Arnd Christian Konig

摘要:

本文建立了b位最小哈希算法的理论框架。原始的最小散列方法已经成为用于估计集合相似性的标准技术(例如,相似性)与信息检索、数据管理、社交网络和计算广告中的应用。通过仅存储每个(minwise)散列值的最低$b$位(例如,b=1或2),可以在计算效率和存储空间方面获得实质性的优点。我们证明了基本的理论结果,并提供了一个无偏估计的相似任何b。我们证明,即使在最不利的情况下,如果对相似度> 0.5感兴趣,与使用b=64(或b=32)相比,使用b=1可以至少减少21.3(或10.7)的存储空间。

英文摘要:

This paper establishes the theoretical framework of b-bit minwise hashing. The original minwise hashing method has become a standard technique for estimating set similarity (e.g., resemblance) with applications in information retrieval, data management, social networks and computational advertising. By only storing the lowest $b$ bits of each (minwise) hashed value (e.g., b=1 or 2), one can gain substantial advantages in terms of computational efficiency and storage space. We prove the basic theoretical results and provide an unbiased estimator of the resemblance for any b. We demonstrate that, even in the least favorable scenario, using b=1 may reduce the storage space at least by a factor of 21.3 (or 10.7) compared to using b=64 (or b=32), if one is interested in resemblance > 0.5.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑