arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1807.07635cs.DS
收藏

快速两两求和的多分辨率哈希算法

Multi-Resolution Hashing for Fast Pairwise Summations

Moses Charikar, Paris Siminelakis

摘要:

在大规模数据集分析中,一个基本的计算原语是对大量对象上的简单函数求和。现代应用程序提出了一个额外的挑战,因为这样的功能往往取决于一个参数向量$y$(查询)是未知的先验。给定一组点$X\subset \mathbb{R}^{d}$和一个成对函数$w:\mathbb{R}^{d}\times \mathbb{R}^{d}\to [0,1]$,我们研究了设计一个数据结构的问题,该数据结构使任何查询$y\in \mathbb{R}^{d}$的求和$Z_{w}(y)=\frac{1}{|X|}\sum_{x\in X}w(x,y)$的次线性时间近似。通过将调和分析(单位和近似理论的分区)的思想与基于哈希的估计器[Charikar,Siminelakis FOCS'17]相结合,我们提供了一个通过哈希设计这种数据结构的通用框架,远远超出了以前的技术所允许的范围。一个关键的设计原则是一个集合的$T\geq 1$散列方案与冲突概率$p_{1},\ldots, p_{T}$,使得$\sup_{t\in [T]}\{p_{t}(x,y)\} = Θ(\sqrt{w(x,y)})$。这导致使用来自每个散列族的子线性数量的样本来近似$Z_{w}(y)$的数据结构。使用这个新的框架以及距离敏感哈希[Aumuller,Christiani,Pagh,Silvestri PODS'18],我们证明了这样的集合可以有效地构建和评估单位球面上内积的任何对数凸函数$w(x,y)=e^{ϕ(\langle x,y\rangle)}$ $x,y\in \mathcal{S}^{d-1}$。我们的方法导致数据结构的次线性查询时间显着提高随机采样,并可用于核密度或分区函数估计。我们提供了我们的结果从球到$\mathbb{R}^{d}$和从标量函数到向量函数的扩展。

英文摘要:

A basic computational primitive in the analysis of massive datasets is summing simple functions over a large number of objects. Modern applications pose an additional challenge in that such functions often depend on a parameter vector $y$ (query) that is unknown a priori. Given a set of points $X\subset \mathbb{R}^{d}$ and a pairwise function $w:\mathbb{R}^{d}\times \mathbb{R}^{d}\to [0,1]$, we study the problem of designing a data-structure that enables sublinear-time approximation of the summation $Z_{w}(y)=\frac{1}{|X|}\sum_{x\in X}w(x,y)$ for any query $y\in \mathbb{R}^{d}$. By combining ideas from Harmonic Analysis (partitions of unity and approximation theory) with Hashing-Based-Estimators [Charikar, Siminelakis FOCS'17], we provide a general framework for designing such data structures through hashing that reaches far beyond what previous techniques allowed. A key design principle is a collection of $T\geq 1$ hashing schemes with collision probabilities $p_{1},\ldots, p_{T}$ such that $\sup_{t\in [T]}\{p_{t}(x,y)\} = Θ(\sqrt{w(x,y)})$. This leads to a data-structure that approximates $Z_{w}(y)$ using a sub-linear number of samples from each hash family. Using this new framework along with Distance Sensitive Hashing [Aumuller, Christiani, Pagh, Silvestri PODS'18], we show that such a collection can be constructed and evaluated efficiently for any log-convex function $w(x,y)=e^{ϕ(\langle x,y\rangle)}$ of the inner product on the unit sphere $x,y\in \mathcal{S}^{d-1}$. Our method leads to data structures with sub-linear query time that significantly improve upon random sampling and can be used for Kernel Density or Partition Function Estimation. We provide extensions of our result from the sphere to $\mathbb{R}^{d}$ and from scalar functions to vector functions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑