分级Bin缓冲:动态外部存储器阵列的在线局部矩
Hierarchical Bin Buffering: Online Local Moments for Dynamic External Memory Arrays
摘要:
局部矩用于局部回归,以计算统计测量,如总和,平均值和标准差,以及近似概率分布。我们考虑的情况下,数据源是一个非常大的I/O数组的大小为n,我们要计算的前N个局部矩,为某个常数N。时间复杂度为O(n)。我们开发了一系列的算法越来越复杂,使用预先计算和额外的缓冲区空间,以加快查询。更简单的算法将I/O数组划分为称为bin的连续范围,它们不仅适用于局部矩查询,而且适用于代数查询(MAX,AVERAGE,SUM等)。对于大小为sqrt{n}的N个缓冲区,时间复杂度下降到O(sqrt n)。更复杂的方法使用分级缓冲,并且当使用大小为n/b的N个分级缓冲器时,具有对数时间复杂度(O(b log_b n))。使用重叠箱缓冲,我们表明,只有一个单一的缓冲区是必要的,与基于小波的算法,但使用更少的存储。应用存在于多维和统计数据库中的海量数据集、交互式图像处理和可视化。
英文摘要:
Local moments are used for local regression, to compute statistical measures such as sums, averages, and standard deviations, and to approximate probability distributions. We consider the case where the data source is a very large I/O array of size n and we want to compute the first N local moments, for some constant N. Without precomputation, this requires O(n) time. We develop a sequence of algorithms of increasing sophistication that use precomputation and additional buffer space to speed up queries. The simpler algorithms partition the I/O array into consecutive ranges called bins, and they are applicable not only to local-moment queries, but also to algebraic queries (MAX, AVERAGE, SUM, etc.). With N buffers of size sqrt{n}, time complexity drops to O(sqrt n). A more sophisticated approach uses hierarchical buffering and has a logarithmic time complexity (O(b log_b n)), when using N hierarchical buffers of size n/b. Using Overlapped Bin Buffering, we show that only a single buffer is needed, as with wavelet-based algorithms, but using much less storage. Applications exist in multidimensional and statistical databases over massive data sets, interactive image processing, and visualization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。