arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 1909.11564math.STcs.DScs.LGstat.COstat.MLstat.TH
收藏

数据流中不同对象数量的分析置信区间

Analytical confidence intervals for the number of different objects in data streams

Università degli Studi di Milano(米兰大学)

Giacomo Aletti

摘要:

本文提出了一种新的分析统计方法来分析一类Flajolet-Martin算法(FMa),并给出了流中不同元素个数F0的解析置信区间. FMa类在大数据流学习中已经非常流行,并且文献的关注主要基于算法方面,基本上是复杂度最优性,而这些算法的统计分析通常是抽象地面对的。这里提供的分析显示了与数学特殊函数和极值理论的深刻联系。后一种联系可能有助于解释启发式的考虑,而第一个打开许多数字问题,面临着在本文件的结尾。最后,算法进行了测试,匿名的真实数据流和蒙特卡洛模拟提供支持我们的分析选择在这种情况下。

英文摘要:

This paper develops a new mathematical-statistical approach to analyze a class of Flajolet-Martin algorithms (FMa), and provides analytical confidence intervals for the number F0 of distinct elements in a stream, based on Chernoff bounds. The class of FMa has reached a significant popularity in bigdata stream learning, and the attention of the literature has mainly been based on algorithmic aspects, basically complexity optimality, while the statistical analysis of these class of algorithms has been often faced heuristically. The analysis provided here shows deep connections with mathematical special functions and with extreme value theory. The latter connection may help in explaining heuristic considerations, while the first opens many numerical issues, faced at the end of the present paper. Finally, the algorithms are tested on an anonymized real data stream and MonteCarlo simulations are provided to support our analytical choice in this context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑