arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.17039cs.IR
收藏

通过块级嵌入和Top-k交互细化实现高效的长文档重排序

Efficient Long-Document Reranking via Block-Level Embeddings and Top-k Interaction Refinement

Soochow University(苏州大学);Univ Grenoble Alpes, CNRS, Grenoble INP, LIG(格勒诺布尔阿尔卑斯大学,法国国家科学研究中心,格勒诺布尔理工学院)

Minghan Li, Eric Gaussier, Guodong Zhou

AI总结:

本文提出基于块级嵌入和Top-k交互细化的高效长文档重排序方法,通过块级表示和轻量级模块提升重排序效果并保持低延迟。

摘要:

密集编码器和基于LLM的重排序器在处理长文档时存在困难:单向量表示会稀释细粒度的相关性,而交叉编码器往往在实际重排序中成本过高。我们提出了一种基于块级嵌入的高效长文档重排序框架。每个文档被分割成短块并编码为块嵌入,这些嵌入可以预先计算。给定一个查询,我们对其进行一次编码,并通过聚合top-k查询块相似性并使用简单的加权和对每个候选文档进行评分,从而获得强且可解释的块级相关性信号。为了捕捉所选块之间的依赖关系并抑制冗余,我们引入了Top-k交互细化(TIR),这是一种轻量级的集间模块,它在top-k块上应用查询条件的注意力,并产生一个有界的残差修正来修正块分数。TIR只引入少量参数,并在top-k块上操作,保持查询时间的开销较低。在长文档重排序基准(TREC DL和MLDR-zh)上的实验表明,块表示在单向量编码器上显著改进,而TIR在强长文档重排序基线上提供了一致的额外增益,同时保持实用的重排序延迟。例如,在TREC DL 2023上,使用相同块预算k=65时,NDCG@10从0.395提高到0.451,最多使用4095个标记。所得到的模型通过暴露哪些块驱动每个文档的分数以及如何细化重新分配它们的贡献来支持可解释性。

英文摘要:

Dense encoders and LLM-based rerankers struggle with long documents: single-vector representations dilute fine-grained relevance, while cross-encoders are often too expensive for practical reranking. We present an efficient long-document reranking framework based on block-level embeddings. Each document is segmented into short blocks and encoded into block embeddings that can be precomputed offline. Given a query, we encode it once and score each candidate document by aggregating top-k query-block similarities with a simple weighted sum, yielding a strong and interpretable block-level relevance signal. To capture dependencies among the selected blocks and suppress redundancy, we introduce Top-k Interaction Refinement (TIR), a lightweight setwise module that applies query-conditioned attention over the top-k blocks and produces a bounded residual correction to block scores. TIR introduces only a small number of parameters and operates on top-k blocks, keeping query-time overhead low. Experiments on long-document reranking benchmarks (TREC DL and MLDR-zh) show that block representations substantially improve over single-vector encoders, and TIR provides consistent additional gains over strong long-document reranking baselines while maintaining practical reranking latency. For example, on TREC DL 2023, NDCG at 10 improves from 0.395 to 0.451 with the same block budget k = 65, using at most 4095 tokens. The resulting model supports interpretability by exposing which blocks drive each document's score and how refinement redistributes their contributions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑