基于语言模型提示的少样本多跳问答重排序
Few-shot Reranking for Multi-hop QA via Language Model Prompting
AI总结：
为解决多跳问答检索器训练依赖大量标注数据的问题，提出基于大语言模型提示的少样本重排序方法PromptRank，仅用128个样本即在HotpotQA上取得强劲检索性能。
摘要：
我们研究开放域问题下多跳QA的少样本重排序。为缓解检索器训练需要大量标注问题-文档对的需求，我们提出PromptRank，它依赖大语言模型提示进行多跳路径重排序。PromptRank首先构建一个包含候选文档路径的指令式提示，然后根据语言模型在给定路径提示下问题的条件似然，计算给定问题与该路径之间的相关性分数。在HotpotQA上，与在数千个样本上训练的最先进方法相比，PromptRank仅用128个训练样本便取得了强劲的检索性能——PromptRank的recall@10为73.6，而PathRetriever为77.8，多跳密集检索为77.5。代码可在https://github.com/mukhal/PromptRank获取。
英文摘要：
We study few-shot reranking for multi-hop QA with open-domain questions. To alleviate the need for a large number of labeled question-document pairs for retriever training, we propose PromptRank, which relies on large language models prompting for multi-hop path reranking. PromptRank first constructs an instruction-based prompt that includes a candidate document path and then computes the relevance score between a given question and the path based on the conditional likelihood of the question given the path prompt according to a language model. PromptRank yields strong retrieval performance on HotpotQA with only 128 training examples compared to state-of-the-art methods trained on thousands of examples -- 73.6 recall@10 by PromptRank vs. 77.8 by PathRetriever and 77.5 by multi-hop dense retrieval. Code available at https://github.com/mukhal/PromptRank
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
