arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.09933cs.IR
收藏

基于Voronoi单元的原理化令牌修剪方法:用于晚期交互检索模型

A Voronoi Cell Formulation for Principled Token Pruning in Late-Interaction Retrieval Models

Université Sorbonne Paris Nord, CNRS, LIPN(索邦巴黎北大学,法国国家科学研究中心,LIPN);Sorbonne Université, CNRS, ISIR(索邦大学,法国国家科学研究中心,ISIR)

Yash Kankanampati, Yuxuan Zong, Nadi Tomeh, Benjamin Piwowarski, Joseph Le Roux

AI总结:

本文提出基于超空间几何的Voronoi单元方法,用于原理化修剪晚期交互检索模型中的令牌嵌入,通过保留检索质量的同时减少索引存储开销。

摘要:

晚期交互模型如ColBERT在各种检索任务中表现优异,但需要为每个文档令牌存储密集嵌入,导致显著的索引存储开销。以往工作通过统计和经验措施尝试修剪低重要性令牌嵌入,但往往缺乏正式理论基础或效果不佳。为解决这些问题,我们引入基于超空间几何的框架,将令牌修剪视为嵌入空间中的Voronoi单元估计问题。通过将每个令牌的影响解释为其Voronoi区域的度量,我们的方法实现了保留检索质量的同时减少索引大小的原理化修剪。通过实验,我们证明这种方法不仅是一种具有竞争力的修剪策略,还是一种有价值的工具,用于改进和解释密集检索系统中令牌层面的行为。

英文摘要:

Late-interaction models such as ColBERT offer competitive performance across various retrieval tasks but require storing a dense embedding for each document token, leading to a substantial index storage overhead. Past works address this by attempting to prune low-importance token embeddings based on statistical and empirical measures, but they often either lack formal grounding or are ineffective. To address these shortcomings, we introduce a framework grounded in hyperspace geometry and cast token pruning as a Voronoi cell estimation problem in the embedding space. By interpreting each token's influence as a measure of its Voronoi region, our approach enables principled pruning that retains retrieval quality while reducing index size. Through our experiments, we demonstrate that this approach serves not only as a competitive pruning strategy but also as a valuable tool for improving and interpreting token-level behavior within dense retrieval systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑