基于Gumbel分布的基数估计
Cardinality estimation using Gumbel distribution
摘要：
基数估计是一项任务，在一个可能有重复元素的大型数据集中近似不同元素的数量。LogLog和HyperLogLog（参见Durand和Flajolet [ESA 2003]，Flajolet等人[Discrete Math Theor. 2007]）是用于基数估计的小空间草图方案，其既具有强有力的理论性能保证，又在实践中是高度有效的。这使得它们成为一个非常受欢迎的解决方案，在大数据系统中有许多实现（例如Algebird，Apache DataSketches，BigQuery，Presto和Redis）。然而，尽管具有简单而优雅的公式，但LogLog和HyperLogLog的分析都非常复杂--跨越了数十页的分析组合学和复变函数分析。我们提出了一个修改LogLog和HyperLogLog，用连续的Gumbel分布取代离散的几何分布。这导致对估计保证的非常简短、简单和基本的分析，以及估计器的更平滑的行为。
英文摘要：
Cardinality estimation is the task of approximating the number of distinct elements in a large dataset with possibly repeating elements. LogLog and HyperLogLog (c.f. Durand and Flajolet [ESA 2003], Flajolet et al. [Discrete Math Theor. 2007]) are small space sketching schemes for cardinality estimation, which have both strong theoretical guarantees of performance and are highly effective in practice. This makes them a highly popular solution with many implementations in big-data systems (e.g. Algebird, Apache DataSketches, BigQuery, Presto and Redis). However, despite having simple and elegant formulation, both the analysis of LogLog and HyperLogLog are extremely involved -- spanning over tens of pages of analytic combinatorics and complex function analysis. We propose a modification to both LogLog and HyperLogLog that replaces discrete geometric distribution with a continuous Gumbel distribution. This leads to a very short, simple and elementary analysis of estimation guarantees, and smoother behavior of the estimator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
