arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.05041cs.DS
收藏

面向推土机距离的数据依赖型局部敏感哈希

Data-Dependent LSH for the Earth Mover's Distance

Rajesh Jayaram, Erik Waingarten, Tian Zhang

AI总结:

本研究提出面向推土机距离的新型数据依赖型局部敏感哈希方案,将最近邻搜索近似比从$O(\log^2 s)$提升至$\tilde{O}(\log s)$,并给出最优分布型概要,方案在常数碰撞概率的LSH中近最优。

摘要:

我们提出了面向推土机距离($\textsf{EMD}$)的新型数据依赖型局部敏感哈希方案(LSH),由此将$\textsf{EMD}$下最近邻搜索的最优近似比提升了一个二次因子。此处度量$\textsf{EMD}_s(\reals^d,\text{ell}_p)$的研究对象是$\reals^d$中含$s$个向量的集合,对于任意两个含$s$个向量的集合$x,y$,距离$\textsf{EMD}(x,y)$是$x$与$y$之间完美匹配的最小代价,其中两个向量匹配的代价为它们的$\text{ell}_p$距离。此前,Andoni、Indyk和Krauthgamer针对$p \n [1,2]$的$\textsf{EMD}_s(\reals^d,\text{ell}_p)$提出了一种(数据无关的)局部敏感哈希方案,近似比为$O(\text{log}^2 s)$。通过采用数据依赖的设计,我们将近似比提升至$\tilde{O}(\text{log} s)$。\n我们的核心技术贡献在于证明:对于任意支撑在度量$\textsf{EMD}_s(\reals^d, \text{ell}_p)$上的分布$\u03bc$,存在针对$\u03bc$稠密区域的数据依赖型LSH,可达到$\tilde{O}(\text{log} s)$的近似比;而数据无关型LSH在这些稠密区域之外,实际上也能达到$\tilde{O}(\text{log} s)$的近似比。最后,我们展示了如何在不产生额外近似损失的前提下,将这两种哈希方案“拼接”在一起。\n除最近邻搜索外,我们的数据依赖型LSH还能给出推土机距离的最优(分布型)概要。根据已知的概要下界,这意味着在所有能以常数概率让近邻点发生碰撞的LSH中,我们的方案是最优的(误差至多为$\text{poly}(\text{log log} s)$因子)。

英文摘要:

We give new data-dependent locality sensitive hashing schemes (LSH) for the Earth Mover's Distance ($\mathsf{EMD}$), and as a result, improve the best approximation for nearest neighbor search under $\mathsf{EMD}$ by a quadratic factor. Here, the metric $\mathsf{EMD}_s(\mathbb{R}^d,\ell_p)$ consists of sets of $s$ vectors in $\mathbb{R}^d$, and for any two sets $x,y$ of $s$ vectors the distance $\mathsf{EMD}(x,y)$ is the minimum cost of a perfect matching between $x,y$, where the cost of matching two vectors is their $\ell_p$ distance. Previously, Andoni, Indyk, and Krauthgamer gave a (data-independent) locality-sensitive hashing scheme for $\mathsf{EMD}_s(\mathbb{R}^d,\ell_p)$ when $p \in [1,2]$ with approximation $O(\log^2 s)$. By being data-dependent, we improve the approximation to $\tilde{O}(\log s)$. Our main technical contribution is to show that for any distribution $μ$ supported on the metric $\mathsf{EMD}_s(\mathbb{R}^d, \ell_p)$, there exists a data-dependent LSH for dense regions of $μ$ which achieves approximation $\tilde{O}(\log s)$, and that the data-independent LSH actually achieves a $\tilde{O}(\log s)$-approximation outside of those dense regions. Finally, we show how to "glue" together these two hashing schemes without any additional loss in the approximation. Beyond nearest neighbor search, our data-dependent LSH also gives optimal (distributional) sketches for the Earth Mover's Distance. By known sketching lower bounds, this implies that our LSH is optimal (up to $\mathrm{poly}(\log \log s)$ factors) among those that collide close points with constant probability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑