arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.01668cs.DS

相对误差流式分位数

Relative Error Streaming Quantiles

Graham Cormode, Zohar Karnin, Edo Liberty, Justin Thaler, Pavel Veselý

AI总结:

针对流式数据秩估计的乘性误差需求,本文提出一种空间效率更优的随机化概要,其空间界接近最优,无需预知流长度且可完全合并,适用于并行分布式环境。

摘要:

在流式数据上估计秩、分位数和分布是数据分析与监控中的核心任务。给定来自具备全序关系的数据全集的包含$n$个元素的流,该任务的目标是计算大小为$n$的多重对数级的概要(数据结构)。基于该概要与查询项$y$,应当能够近似其在流中的秩,即流中小于或等于$y$的元素数量。\n 迄今为止的大多数研究都聚焦于加性$\boldsymbol{\varepsilon n}$误差近似,其中KLL概要达到了最优的渐近行为,是该方向的代表性成果。本文研究秩的乘性$(1\pm\varepsilon)$误差近似。乘性误差的实际动机源于理解分布尾部的需求,因此要求概要在极值附近具备更高的精度。\n 现有研究中空间效率最高的算法存储的全集元素数量为$O(\log(\varepsilon^2 n)/\varepsilon^2)$或$O(\log^3(\varepsilon n)/\varepsilon)$。我们提出了一种存储$O(\log^{1.5}(\varepsilon n)/\varepsilon)$个元素的随机化概要,它能够以高常数概率对每个全集元素的秩进行$(1\pm\varepsilon)$近似;该空间界与最优值的差距在$O(\sqrt{\log(\varepsilon n)})$因子以内。我们的算法无需预先知晓流的长度,并且具备完全可合并性,因此适用于并行与分布式计算环境。

英文摘要:

Estimating ranks, quantiles, and distributions over streaming data is a central task in data analysis and monitoring. Given a stream of $n$ items from a data universe equipped with a total order, the task is to compute a sketch (data structure) of size polylogarithmic in $n$. Given the sketch and a query item $y$, one should be able to approximate its rank in the stream, i.e., the number of stream elements smaller than or equal to $y$. Most works to date focused on additive $\varepsilon n$ error approximation, culminating in the KLL sketch that achieved optimal asymptotic behavior. This paper investigates multiplicative $(1\pm\varepsilon)$-error approximations to the rank. Practical motivation for multiplicative error stems from demands to understand the tails of distributions, and hence for sketches to be more accurate near extreme values. The most space-efficient algorithms due to prior work store either $O(\log(\varepsilon^2 n)/\varepsilon^2)$ or $O(\log^3(\varepsilon n)/\varepsilon)$ universe items. We present a randomized sketch storing $O(\log^{1.5}(\varepsilon n)/\varepsilon)$ items that can $(1\pm\varepsilon)$-approximate the rank of each universe item with high constant probability; this space bound is within an $O(\sqrt{\log(\varepsilon n)})$ factor of optimal. Our algorithm does not require prior knowledge of the stream length and is fully mergeable, rendering it suitable for parallel and distributed computing environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑