基数估计满足良图灵
Cardinality Estimation Meets Good-Turing
摘要:
基数估计算法接收顺序可能是任意的元素流,可能有重复,并返回不同元素的数量。这种算法通常寻求以输出不准确为代价来最小化所需的存储和处理。这些算法的实际应用需要处理大量的监控数据,这使得收集和分析整个输入流变得不切实际。在这种情况下,通常的做法是只对一小部分流元素进行采样和处理。本文提出并分析了一种将各种基数估计算法与抽样过程相结合的通用算法。我们证明了所提出的抽样算法不影响估计量的渐近无偏性,并分析了抽样对估计量方差的影响。
英文摘要:
Cardinality estimation algorithms receive a stream of elements whose order might be arbitrary, with possible repetitions, and return the number of distinct elements. Such algorithms usually seek to minimize the required storage and processing at the price of inaccuracy in their output. Real-world applications of these algorithms are required to process large volumes of monitored data, making it impractical to collect and analyze the entire input stream. In such cases, it is common practice to sample and process only a small part of the stream elements. This paper presents and analyzes a generic algorithm for combining every cardinality estimation algorithm with a sampling process. We show that the proposed sampling algorithm does not affect the estimator's asymptotic unbiasedness, and we analyze the sampling effect on the estimator's variance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。