arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.00807cs.IRcs.CL
收藏

UDAPDR:通过LLM提示与重排序器蒸馏实现无监督域适应

UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers

Stanford University(斯坦福大学);IBM Research AI(IBM人工智能研究院)

Jon Saad-Falcon, Omar Khattab, Keshav Santhanam, Radu Florian, Martin Franz, Salim Roukos, Avirup Sil, Md Arafat Sultan, Christopher Potts

AI总结:

针对信息检索中标注数据稀缺和域偏移问题,提出用LLM生成合成查询微调重排序器并蒸馏为高效检索器的方法,提升长尾域零样本准确率并降低延迟。

摘要:

许多信息检索任务需要大量标注数据集进行微调。然而,此类数据集往往不可用,且由于域偏移,其在实际应用中的效用会迅速降低。为解决这一挑战,我们开发并论证了一种利用大型语言模型(LLMs)廉价生成大量合成查询的方法。该方法首先使用昂贵的LLM生成少量合成查询,随后使用成本低得多的LLM生成大量合成查询,用于微调一系列重排序器模型。这些重排序器随后被蒸馏为一个高效的单检索器,用于目标域。我们证明,该技术提升了长尾域的零样本准确率,并实现了比标准重排序方法显著更低的延迟。

英文摘要:

Many information retrieval tasks require large labeled datasets for fine-tuning. However, such datasets are often unavailable, and their utility for real-world applications can diminish quickly due to domain shifts. To address this challenge, we develop and motivate a method for using large language models (LLMs) to generate large numbers of synthetic queries cheaply. The method begins by generating a small number of synthetic queries using an expensive LLM. After that, a much less expensive one is used to create large numbers of synthetic queries, which are used to fine-tune a family of reranker models. These rerankers are then distilled into a single efficient retriever for use in the target domain. We show that this technique boosts zero-shot accuracy in long-tail domains and achieves substantially lower latency than standard reranking methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑