arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1912.07600cs.DS
收藏

一种新的数据流频率估计方法

A new Frequency Estimation Sketch for Data Streams

Ning Li

摘要:

在数据流应用中,关键问题之一是估计每个项目在特定多集中的出现频率。多集意味着这个集合中的每个项目可以出现多次。许多应用中的数据流都是高速流,包含海量数据,如实时IP流量、图形流、Web点击和抓取、传感器数据库、自然语言处理(NLP)[2][6]等,在这些应用中,需要服务器实时的记录流信息。然而,由于这些应用中的数据流是高速的,准确记录和估计项目频率总是不切实际的。解决这个问题的另一种方法是基于概率数据结构估计项目频率,这种方法已广泛用于高速数据流估计[7][9]。Sketches是一种典型的概率数据结构,最初设计用于估计数据流中的项目频率[10][15]。目前,草图已用于许多不同的场景,例如压缩感知中的稀疏近似[16],自然语言处理[17,18],数据图[19,20]等等[21]。在本文中,我们主要集中在用于频率估计的草图。

英文摘要:

In data stream applications, one of the critical issues is to estimate the frequency of each item in the specific multiset. The multiset means that each item in this set can appear multiple times. The data streams in many applications are high-speed streams which contain massive data, such as real-time IP traffic, graph streams, web clicks and crawls, sensor database, and natural language processing (NLP) [2][6], etc. In these applications, the stream information needs to be recorded by the servers in real time. However, since the data streams in these applications are high-speed, the accurate recording and estimation of item frequencies is always impractical. An alternative approach for addressing this problem is to estimate the item frequencies based on probabilistic data structures, and this approach has been widely used in the high-speed data streams estimation [7][9]. Sketches is one of the typical probabilistic data structures, which are initially designed for the estimation of item frequencies in data streams [10][15]. At present, the sketches have been used in many different scenarios, such as sparse approximation in compressed sensing [16], natural language processing [17, 18], data graph [19, 20], and more [21]. In this paper, we mainly focus on the sketches used for frequency estimation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑