arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.07023cs.IRcs.AI
收藏

用于第一阶段检索的复合代码稀疏自动编码器

Composite Code Sparse Autoencoders for first stage retrieval

Naver Labs Europe(Naver欧洲实验室)

Carlos Lassance, Thibault Formal, Stephane Clinchant

摘要:

我们提出了一个复合代码稀疏自动编码器(CCSA)的近似最近邻(ANN)搜索的基础上Siamese-BERT模型的文件表示的方法。在信息检索(IR)中,排序管道通常被分解为两个阶段:第一阶段集中于从整个集合中检索候选集合。第二阶段依靠更复杂的模型对候选集重新排序。近年来,Siamese-BERT模型已经被用来作为第一阶段的排名,以取代或补充传统的词袋模型。然而,索引和搜索一个大的文档集需要高效的相似性搜索密集的向量,这就是为什么人工神经网络技术发挥作用。由于复合码是自然稀疏的,我们首先展示了CCSA如何学习高效的并行倒排索引由于均匀正则化。第二,CCSA可以用作二进制量化方法,我们建议将其与最近的基于图的人工神经网络技术相结合。我们在MSMARCO数据集上的实验表明,CCSA优于IVF产品量化。此外,CCSA二进制量化对于基于图的HNSW方法的索引大小和存储器使用是有益的,同时保持良好的查全率和MRR水平。第三,我们比较了最近的监督量化方法的图像检索,发现CCSA是能够优于他们。

英文摘要:

We propose a Composite Code Sparse Autoencoder (CCSA) approach for Approximate Nearest Neighbor (ANN) search of document representations based on Siamese-BERT models. In Information Retrieval (IR), the ranking pipeline is generally decomposed in two stages: the first stage focus on retrieving a candidate set from the whole collection. The second stage re-ranks the candidate set by relying on more complex models. Recently, Siamese-BERT models have been used as first stage ranker to replace or complement the traditional bag-of-word models. However, indexing and searching a large document collection require efficient similarity search on dense vectors and this is why ANN techniques come into play. Since composite codes are naturally sparse, we first show how CCSA can learn efficient parallel inverted index thanks to an uniformity regularizer. Second, CCSA can be used as a binary quantization method and we propose to combine it with the recent graph based ANN techniques. Our experiments on MSMARCO dataset reveal that CCSA outperforms IVF with product quantization. Furthermore, CCSA binary quantization is beneficial for the index size, and memory usage for the graph-based HNSW method, while maintaining a good level of recall and MRR. Third, we compare with recent supervised quantization methods for image retrieval and find that CCSA is able to outperform them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑