NLP中近似最近邻搜索的子空间近似
Subspace Approximation for Approximate Nearest Neighbor Search in NLP
摘要:
大多数自然语言处理任务都可以表示为近似最近邻搜索问题,如词语类比、文档相似度、机器翻译等。以问答任务为例,给定一个问题作为查询,目标是在训练数据集中搜索其最近邻作为答案。然而,现有的近似最近邻搜索方法在实际应用中存在以下问题:1)数据中存在噪声; 2)大规模数据集会产生巨大的检索空间和较高的搜索时间复杂度。为了解决这些问题,我们提出了一种新的近似最近邻搜索框架,该框架i)将数据投影到基于子空间的谱分析中,以消除噪声的影响; ii)将训练数据集划分为不同的组,以减少搜索空间。具体地,检索空间从$O(n)$减少到$O(\log n)$(其中$n$是训练数据集中的数据点的数量)。我们证明了在投影子空间中检索到的最近邻与原始特征空间中的最近邻相同。我们展示了我们的框架在现实世界的自然语言处理任务的出色表现。
英文摘要:
Most natural language processing tasks can be formulated as the approximated nearest neighbor search problem, such as word analogy, document similarity, machine translation. Take the question-answering task as an example, given a question as the query, the goal is to search its nearest neighbor in the training dataset as the answer. However, existing methods for approximate nearest neighbor search problem may not perform well owing to the following practical challenges: 1) there are noise in the data; 2) the large scale dataset yields a huge retrieval space and high search time complexity. In order to solve these problems, we propose a novel approximate nearest neighbor search framework which i) projects the data to a subspace based spectral analysis which eliminates the influence of noise; ii) partitions the training dataset to different groups in order to reduce the search space. Specifically, the retrieval space is reduced from $O(n)$ to $O(\log n)$ (where $n$ is the number of data points in the training dataset). We prove that the retrieved nearest neighbor in the projected subspace is the same as the one in the original feature space. We demonstrate the outstanding performance of our framework on real-world natural language processing tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。