关于Top-K串的最优检索
On Optimal Top-K String Retrieval
摘要:
令${\cal{D}}$ = $\{d_1, d_2, d_3, ..., d_D\}$是总长度为$n$的给定的$D$(字符串)文档集合。top-$k$文档检索问题是索引$\cal{D}$,使得当长度为$p$的模式$P$和参数$k$作为查询到来时,索引返回与模式$P$最相关的$k$文档。Hon et. \cite{HSV09}等人给出了第一个线性空间框架来解决$O(p + k\log k)$时间上的这个问题。Navarro和Nekrich \cite{NN12}将其改进为$O(p + k)$。这些结果足够强大,可以支持任意相关函数,如频率,接近度,PageRank等。在桌面或电子邮件搜索等许多应用程序中,数据驻留在磁盘上,因此需要磁盘绑定索引。尽管这个问题在理论、实践和压缩方面不断取得进展,但到目前为止,外部存储模型中的任何非平凡边界都是难以捉摸的。内部存储器(或RAM)解决这个问题分解成$O(p)$的子问题,从而导致$O(p)$的附加因子。在外部存储器中,这些方法将导致$O(p)$ I/O而不是最佳$O(p/B)$ I/O项,其中$B$是块大小。我们重新解释的问题独立的$p$,作为间隔刺优先于树形结构。这导致我们在外部存储器中的线性空间索引,其支持对于任何恒定$h$ { $\log^{(1)}n =\log n$和$\log^{(h)} n = \log (\log^{(h-1)} n)$ }在接近最优的$O(p/B + \log_B n + \log^{(h)} n + k/B)$ I/O中的顶级$k$查询(具有未排序的输出)。然后得到具有最优$O(p/B+\log_B n + k/B)$ I/O的$O(n\log^*n)$空间索引。
英文摘要:
Let ${\cal{D}}$ = $\{d_1, d_2, d_3, ..., d_D\}$ be a given set of $D$ (string) documents of total length $n$. The top-$k$ document retrieval problem is to index $\cal{D}$ such that when a pattern $P$ of length $p$, and a parameter $k$ come as a query, the index returns the $k$ most relevant documents to the pattern $P$. Hon et. al. \cite{HSV09} gave the first linear space framework to solve this problem in $O(p + k\log k)$ time. This was improved by Navarro and Nekrich \cite{NN12} to $O(p + k)$. These results are powerful enough to support arbitrary relevance functions like frequency, proximity, PageRank, etc. In many applications like desktop or email search, the data resides on disk and hence disk-bound indexes are needed. Despite of continued progress on this problem in terms of theoretical, practical and compression aspects, any non-trivial bounds in external memory model have so far been elusive. Internal memory (or RAM) solution to this problem decomposes the problem into $O(p)$ subproblems and thus incurs the additive factor of $O(p)$. In external memory, these approaches will lead to $O(p)$ I/Os instead of optimal $O(p/B)$ I/O term where $B$ is the block-size. We re-interpret the problem independent of $p$, as interval stabbing with priority over tree-shaped structure. This leads us to a linear space index in external memory supporting top-$k$ queries (with unsorted outputs) in near optimal $O(p/B + \log_B n + \log^{(h)} n + k/B)$ I/Os for any constant $h${$\log^{(1)}n =\log n$ and $\log^{(h)} n = \log (\log^{(h-1)} n)$}. Then we get $O(n\log^*n)$ space index with optimal $O(p/B+\log_B n + k/B)$ I/Os.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。