arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

AdaWidth:面向密集检索的查询自适应嵌入宽度

AdaWidth: Query-Adaptive Embedding Width for Dense Retrieval

Shubing Yang, Dongfang Zhao

arXiv 2608.23862首次发表:更新:

AI 中文总结

AdaWidth通过正交前缀适配器和轻量级路由器,使密集检索的查询自适应评估维度数,在6项任务和5个冻结编码器上,以更少维度达到当前最先进方法的NDCG@10性能。

AI 中文摘要

高维嵌入是密集检索的核心,但并非所有维度都需要在检索时进行评估。现有方法通过两种方式降低维度:对所有查询截断相同的前导维度,或为每个查询掩码不同的维度子集,但仍需存储和访问完整嵌入。然而,同一任务内的查询在实现排名稳定所需的维度数量上差异显著。我们提出AdaWidth,它在共享前缀表示下使评估维度数适配每个查询。正交前缀适配器对查询和文档应用单一学习到的旋转,将判别信号集中在前导坐标,同时保持全宽度内积不变。轻量级路由器会读取查询已生成排名的顺序统计量,仅对那些前几名结果会发生变化的查询评估更多维度。我们进一步推导前缀充分性分析,表明所需维度数由检索截止时的竞争文档决定:其随语料库大小对数增长,随检索深度对数下降,且在所有查询中呈现重尾分布。在6个检索任务和5个冻结编码器上,AdaWidth以每个查询减少55%至84%的维度数,达到了当前最先进的维度缩减方法的NDCG@10指标。

英文摘要

High-dimensional embeddings are central to dense retrieval, but not all of these dimensions need to be evaluated at retrieval time. Existing methods reduce dimensions in two ways: truncating the same leading dimensions for every query, or masking a different subset for each query while still storing and accessing the full embedding. Yet queries within a single task differ widely in the number of dimensions they need for their rankings to stabilize. We introduce AdaWidth, which adapts the number of evaluated dimensions to each query within a shared prefix representation. An orthogonal prefix adapter applies a single learned rotation to queries and documents alike, concentrating discriminative signal in leading coordinates while leaving every full width inner product unchanged. A lightweight router then reads order statistics off the ranking a query has already produced, and evaluates more dimensions only for the queries whose top results would change. We further derive a prefix sufficiency analysis showing that the required number of dimensions is set by the competing documents at the retrieval cutoff: it grows logarithmically with corpus size, decreases logarithmically with retrieval depth, and remains heavy-tailed across queries. Across six retrieval tasks and five frozen encoders, AdaWidth matches the NDCG@10 of state-of-the-art dimensionality reduction using 55% to 84% fewer dimensions per query.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑