arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12663cs.CLcs.IRcs.LG
收藏

SoftQE:由LLM扩展的查询的学习表示

SoftQE: Learned Representations of Queries Expanded by LLMs

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);Alexa AI, Amazon(亚马逊Alexa AI)

Varad Pimpalkhute, John Heyer, Xusen Yin, Sameer Gupta

AI总结:

SoftQE通过将输入查询嵌入映射到LLM扩展查询嵌入,在不增加推理延迟和成本的前提下利用LLM知识改进稠密检索,并在多个域外BEIR任务上平均提升2.83个百分点。

摘要:

我们研究了将大型语言模型(LLMs)集成到查询编码器中,以在不增加延迟和成本的情况下改进稠密检索,方法是在推理时规避对LLMs的依赖。SoftQE通过将输入查询的嵌入映射到LLM扩展查询的嵌入来融入LLMs的知识。虽然在域内MS-MARCO指标上相对于多个强基线的改进有限,但SoftQE在五个域外BEIR任务上平均将性能提高了2.83个绝对百分点。

英文摘要:

We investigate the integration of Large Language Models (LLMs) into query encoders to improve dense retrieval without increasing latency and cost, by circumventing the dependency on LLMs at inference time. SoftQE incorporates knowledge from LLMs by mapping embeddings of input queries to those of the LLM-expanded queries. While improvements over various strong baselines on in-domain MS-MARCO metrics are marginal, SoftQE improves performance by 2.83 absolute percentage points on average on five out-of-domain BEIR tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑