QPOPSS:面向频繁流元素查找的查询与并行优化 Space-Saving 算法
QPOPSS: Query and Parallelism Optimized Space-Saving for Finding Frequent Stream Elements
AI总结:
QPOPSS 通过快速查询的 Space-Saving 实现与细粒度线程同步,解决了流数据频繁元素查找中并发更新与查询的算法保证问题,实现线性吞吐量扩展、最高准确率和极低内存占用。
摘要:
频繁元素问题是高要求流数据分析中的关键组成部分,涉及选取出现次数超过用户指定阈值的元素。快速、内存高效的 ε-近似概要算法能够选出所有频繁元素,但可能根据 ε(用户定义参数)对它们产生过高估计。不断演进的应用要求只有通过并行化才能实现的性能。然而,关于并发更新和查询的算法保证一直被忽视。我们提出查询与并行优化的 Space-Saving(QPOPSS),提供并发保证。该设计包括一个支持快速查询的 Space-Saving 算法实现,意味着与并发更新的重叠最小。QPOPSS 将其与工作分配和线程间细粒度同步相结合,在高吞吐量、高准确率和低内存消耗之间迅速取得平衡。我们在各种并发和数据分布条件下的分析展示了空间和近似界限。我们相对于代表性最先进方法的实证评估表明,QPOPSS 的多线程吞吐量线性扩展,同时保持最高准确率,并且内存占用小几个数量级。
英文摘要:
The frequent elements problem, a key component in demanding stream-data analytics, involves selecting elements whose occurrence exceeds a user-specified threshold. Fast, memory-efficient $ε$-approximate synopsis algorithms select all frequent elements but may overestimate them depending on $ε$ (user-defined parameter). Evolving applications demand performance only achievable by parallelization. However, algorithmic guarantees concerning concurrent updates and queries have been overlooked. We propose Query and Parallelism Optimized Space-Saving (QPOPSS), providing concurrency guarantees. The design includes an implementation of the \emph{Space-Saving} algorithm supporting fast queries, implying minimal overlap with concurrent updates. QPOPSS integrates this with the distribution of work and fine-grained synchronization among threads, swiftly balancing high throughput, high accuracy, and low memory consumption. Our analysis, under various concurrency and data distribution conditions, shows space and approximation bounds. Our empirical evaluation relative to representative state-of-the-art methods reveals that QPOPSS's multi-threaded throughput scales linearly while maintaining the highest accuracy, with orders of magnitude smaller memory footprint.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。