arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.09983cs.IR
收藏

基于矩阵分解的稠密检索静态剪枝

Static Pruning in Dense Retrieval using Matrix Decomposition

Federico Siciliano, Francesca Pezzuti, Nicola Tonellotto, Fabrizio Silvestri

AI总结:

针对现有稠密检索嵌入降维方法查询相关、无法离线应用影响效率的问题,提出基于PCA的查询无关静态剪枝方法,可离线执行,降维超50%时NDCG@10下降最多5%,显著提升效率且效果影响极小。

摘要:

在稠密检索时代,文档索引与检索主要依赖将文本文档转换为嵌入向量的编码模型。检索效率与文档数量、嵌入向量规模成正比。近期研究表明,可在不损失检索效果(部分场景下甚至能提升效果)的前提下缩小嵌入向量规模。但这些研究提出的方法是查询相关的,无法离线应用,且需在查询处理阶段进行额外计算,反而会对检索效率造成负面影响。本文提出一种基于主成分分析(Principal Components Analysis,PCA)的新型静态剪枝方法,用于降低嵌入向量的维度。该方法与查询无关,可离线执行,能显著提升稠密检索效率,且对系统效果的影响可忽略不计。实验结果显示,针对不同的稠密检索模型,所提方法可将文档表示的维度降低50%以上,而NDCG@10的下降幅度最高仅为5%。

英文摘要:

In the era of dense retrieval, document indexing and retrieval is largely based on encoding models that transform text documents into embeddings. The efficiency of retrieval is directly proportional to the number of documents and the size of the embeddings. Recent studies have shown that it is possible to reduce embedding size without sacrificing - and in some cases improving - the retrieval effectiveness. However, the methods introduced by these studies are query-dependent, so they can't be applied offline and require additional computations during query processing, thus negatively impacting the retrieval efficiency. In this paper, we present a novel static pruning method for reducing the dimensionality of embeddings using Principal Components Analysis. This approach is query-independent and can be executed offline, leading to a significant boost in dense retrieval efficiency with a negligible impact on the system effectiveness. Our experiments show that our proposed method reduces the dimensionality of document representations by over 50% with up to a 5% reduction in NDCG@10, for different dense retrieval models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑