arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1701.04148cs.DS
收藏

SF-sketch:数据流的两阶段Sketch

SF-sketch: A Two-stage Sketch for Data Streams

Tong Yang, Lingtong Liu, Yibo Yan, Muhammad Shahzad, Yulong Shen, Xiaoming Li, Bin Cui, Gaogang Xie

摘要:

草图是一种概率数据结构,用于记录多集中项目的频率。草图广泛应用于各个领域,特别是涉及处理和存储数据流的领域。在具有高数据率的流式传输应用中,草图非常快地“填满”。因此,它的内容会定期传输到负责回答查询的远程收集器。在本文中,我们提出了一种新的草图,称为瘦胖(SF)草图,它具有显着更高的精度相比,现有技术,一个更小的内存占用,并在同一时间达到相同的速度作为最好的现有草图。我们提出的SF-草图背后的关键思想是维护两个独立的草图:一个称为Slim-subsketch的小草图和一个称为Fat-subsketch的大草图。Slim子草图定期传输到远程收集器,以快速准确地回答查询。但是,胖子草图不会传输到远程收集器,因为它仅用于在插入和删除过程中辅助瘦子草图,而不用于回答查询。我们实现并广泛评估了SF-草图以及之前的几个草图,并将它们并排进行比较。我们的实验结果表明,SF-草图优于最广泛使用的CM-草图高达33.1倍的准确性。我们已经在Github上发布了我们提议的草图以及现有草图的源代码。本文的简短版本将出现在ICDE 2017上。

英文摘要:

A sketch is a probabilistic data structure used to record frequencies of items in a multi-set. Sketches are widely used in various fields, especially those that involve processing and storing data streams. In streaming applications with high data rates, a sketch "fills up" very quickly. Thus, its contents are periodically transferred to the remote collector, which is responsible for answering queries. In this paper, we propose a new sketch, called Slim-Fat (SF) sketch, which has a significantly higher accuracy compared to prior art, a much smaller memory footprint, and at the same time achieves the same speed as the best prior sketch. The key idea behind our proposed SF-sketch is to maintain two separate sketches: a small sketch called Slim-subsketch and a large sketch called Fat-subsketch. The Slim-subsketch is periodically transferred to the remote collector for answering queries quickly and accurately. The Fat-subsketch, however, is not transferred to the remote collector because it is used only to assist the Slim-subsketch during the insertions and deletions and is not used to answer queries. We implemented and extensively evaluated SF-sketch along with several prior sketches and compared them side by side. Our experimental results show that SF-sketch outperforms the most widely used CM-sketch by up to 33.1 times in terms of accuracy. We have released the source codes of our proposed sketch as well as existing sketches at Github. The short version of this paper will appear in ICDE 2017.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑