通过蒸馏将稠密检索器与LLM效用对齐
Aligning Dense Retrievers with LLM Utility via Distillation
AI总结：
提出Utility-Aligned Embeddings (UAE)框架，通过蒸馏LLM的困惑度降低效用分布来训练双编码器，在不增加测试时LLM推理开销的情况下提升稠密检索的精度和效率。
摘要：
稠密向量检索是检索增强生成（RAG）的实用支柱，但相似性搜索可能受限于精度。相反，利用LLM重排序的基于效用的方法通常能实现更优性能，但计算成本高且易受困惑度估计中固有噪声的影响。我们提出Utility-Aligned Embeddings (UAE)，一个旨在将这些优势融合为实用、高性能检索方法的框架。我们将检索表述为分布匹配问题，使用Utility-Modulated InfoNCE目标训练双编码器以模仿由困惑度降低导出的效用分布。该方法将分级效用信号直接注入嵌入空间，无需测试时LLM推理。在QASPER基准上，UAE在召回率@1上提升30.59%，MAP提升30.16%，Token F1提升17.3%，优于强语义基线BGE-Base。关键的是，UAE比高效的LLM重排序方法快180倍以上，同时保持竞争性能，表明将检索与生成效用对齐能在规模上产生可靠的上下文。
英文摘要：
Dense vector retrieval is the practical backbone of Retrieval- Augmented Generation (RAG), but similarity search can suffer from precision limitations. Conversely, utility-based approaches leveraging LLM re-ranking often achieve superior performance but are computationally prohibitive and prone to noise inherent in perplexity estimation. We propose Utility-Aligned Embeddings (UAE), a framework designed to merge these advantages into a practical, high-performance retrieval method. We formulate retrieval as a distribution matching problem, training a bi-encoder to imitate a utility distribution derived from perplexity reduction using a Utility-Modulated InfoNCE objective. This approach injects graded utility signals directly into the embedding space without requiring test-time LLM inference. On the QASPER benchmark, UAE improves retrieval Recall@1 by 30.59%, MAP by 30.16% and Token F1 by 17.3% over the strong semantic baseline BGE-Base. Crucially, UAE is over 180x faster than the efficient LLM re-ranking methods preserving competitive performance, demonstrating that aligning retrieval with generative utility yields reliable contexts at scale.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
