基于大语言模型的伪相关反馈的系统研究
A Systematic Study of Pseudo-Relevance Feedback with LLMs
AI总结：
本文系统研究了基于大语言模型的伪相关反馈方法，发现反馈模型选择和反馈源对PRF效果有重要影响，且从语料库获取反馈在强检索器支持下效果最佳。
摘要：
伪相关反馈（PRF）方法基于大语言模型（LLMs）可以沿着两个关键设计维度进行组织：反馈源，即反馈文本的来源，以及反馈模型，即如何使用给定的反馈文本来细化查询表示。然而，每个维度的独立作用尚不明确，因为两者在实证评估中往往交织在一起。在本文中，我们通过受控实验系统研究了反馈源和反馈模型的选择如何影响PRF的有效性。在13个低资源BEIR任务中使用五种LLM PRF方法，我们的结果表明：（1）反馈模型的选择在PRF有效性中起关键作用；（2）仅从LLM生成的文本中获取的反馈提供了最经济有效的解决方案；（3）当使用强首阶段检索器的候选文档时，从语料库中获取的反馈最为有益。共同，我们的发现为PRF设计空间中哪些元素最为重要提供了更深入的理解。
英文摘要：
Pseudo-relevance feedback (PRF) methods built on large language models (LLMs) can be organized along two key design dimensions: the feedback source, which is where the feedback text is derived from and the feedback model, which is how the given feedback text is used to refine the query representation. However, the independent role that each dimension plays is unclear, as both are often entangled in empirical evaluations. In this paper, we address this gap by systematically studying how the choice of feedback source and feedback model impact PRF effectiveness through controlled experimentation. Across 13 low-resource BEIR tasks with five LLM PRF methods, our results show: (1) the choice of feedback model can play a critical role in PRF effectiveness; (2) feedback derived solely from LLM-generated text provides the most cost-effective solution; and (3) feedback derived from the corpus is most beneficial when utilizing candidate documents from a strong first-stage retriever. Together, our findings provide a better understanding of which elements in the PRF design space are most important.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
