Similarity query is the family of queries based on some similarity metrics. Unlike the traditional database queries which are mostly based on value equality, similarity queries aim to find targets "similar enough to" the given data objects, depending on some similarity metric, e.g., Euclidean distance, cosine similarity and so on. To measure the similarity between data objects, traditional methods normally work on low level or syntax features(e.g., basic visual features on images or bag-of-word features of text), which makes them weak to compute the semantic similarities between objects. So for measuring data similarities semantically, neural embedding is applied. Embedding techniques work by representing the raw data objects as vectors (so called "embeddings" or "neural embeddings" since they are mostly generated by neural network models) that expose the hidden semantics of the raw data, based on which embeddings do show outstanding effectiveness on capturing data similarities, making it one of the most widely used and studied techniques in the state-of-the-art similarity query processing research. But there are still many open challenges on the efficiency of embedding based similarity query processing, which are not so well-studied as the effectiveness. In this survey, we first provide an overview of the "similarity query" and "similarity query processing" problems. Then we talk about recent approaches on designing the indexes and operators for highly efficient similarity query processing on top of embeddings (or more generally, high dimensional data). Finally, we investigate the specific solutions with and without using embeddings in selected application domains of similarity queries, including entity resolution and information retrieval. By comparing the solutions, we show how neural embeddings benefit those applications.
Navigable Proximity Graph-Driven Native Hybrid Queries with Structured and Unstructured Constraints
Hangzhou Dianzi University(杭州电子科技大学)
随着研究兴趣的激增,向量相似性搜索被应用于多个领域,包括数据采掘、计算机视觉和信息检索。{给定一组对象(例如,一组图像)和查询对象,我们可以很容易地将每个对象转换为特征向量,并应用向量相似性搜索来检索最相似的对象。然而,原始向量相似性搜索不能很好地支持\textit{hybrid queries},其中用户不仅输入非结构化查询约束(即,查询对象的特征向量)以及结构化查询约束(即,感兴趣的期望属性)。混合查询处理的目标是识别出与查询对象具有相似特征向量并满足给定属性约束的对象。最近的努力尝试通过分别执行属性过滤和向量相似性搜索并且随后合并结果来回答混合查询,这限制了效率和准确性,因为它们不是为混合查询而专门构建的。本文提出了一种基于邻近图(PG)的原生混合查询(NHQ)框架,为混合查询提供了专用的\textit{composite index and joint pruning}模块。我们可以很容易地部署现有的各种PG在这个框架上有效地处理混合查询。此外,我们提出了两个新的导航PG(NPG)与优化的边缘选择和路由策略,获得更好的整体性能比现有的PG。之后,我们在NHQ中部署了所提出的NPG以形成两种混合查询方法,它们在所有实验数据集上的性能都显着优于最先进的竞争对手(在相同的\textit{Recall}下快10 $\times$),包括8个公共数据集和1个内部真实世界数据集。我们的…