arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.05217cs.CLcs.AIcs.IR
收藏

利用大语言模型的多角色能力实现开放域问答

Harnessing Multi-Role Capabilities of Large Language Models for Open-Domain Question Answering

Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Nankai University(南开大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Moqi, Inc.(墨奇科技);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);University of Electronic Science and Technology of China(电子科技大学)

Hongda Sun, Yuxuan Liu, Chengwei Wu, Haiyu Yan, Cheng Tai, Xin Gao, Shuo Shang, Rui Yan

AI总结:

针对现有开放域问答证据收集范式的不足,提出结合检索与生成优势的LLMQA框架,让大语言模型扮演多角色协同工作,并通过提示优化算法提升效果,在多个基准上实现了答案准确率和证据质量的最优表现。

摘要:

开放域问答(ODQA)已成为信息系统领域的核心研究热点。现有方法主要遵循两种证据收集范式:(1)“检索-阅读”范式,从外部语料库中检索相关文档;(2)“生成-阅读”范式,利用大语言模型(LLM)生成相关文档。但两者都无法完全满足证据的多方面需求。为此,我们提出LLMQA这一通用框架,将ODQA流程划分为查询扩展、文档选择和答案生成三个基本步骤,结合了基于检索和基于生成的证据的优势。鉴于LLM展现出完成各类任务的出色能力,我们指示LLM在框架中扮演生成器、重排器和评估器等多种角色,整合它们在ODQA流程中协同工作。此外,我们提出了一种新颖的提示优化算法,用于优化角色扮演提示,引导LLM生成更高质量的证据和答案。在广泛使用的基准(NQ、WebQ和TriviaQA)上的大量实验结果表明,LLMQA在答案准确率和证据质量方面均达到了最佳性能,展现出其推动ODQA研究与应用的潜力。

英文摘要:

Open-domain question answering (ODQA) has emerged as a pivotal research spotlight in information systems. Existing methods follow two main paradigms to collect evidence: (1) The \textit{retrieve-then-read} paradigm retrieves pertinent documents from an external corpus; and (2) the \textit{generate-then-read} paradigm employs large language models (LLMs) to generate relevant documents. However, neither can fully address multifaceted requirements for evidence. To this end, we propose LLMQA, a generalized framework that formulates the ODQA process into three basic steps: query expansion, document selection, and answer generation, combining the superiority of both retrieval-based and generation-based evidence. Since LLMs exhibit their excellent capabilities to accomplish various tasks, we instruct LLMs to play multiple roles as generators, rerankers, and evaluators within our framework, integrating them to collaborate in the ODQA process. Furthermore, we introduce a novel prompt optimization algorithm to refine role-playing prompts and steer LLMs to produce higher-quality evidence and answers. Extensive experimental results on widely used benchmarks (NQ, WebQ, and TriviaQA) demonstrate that LLMQA achieves the best performance in terms of both answer accuracy and evidence quality, showcasing its potential for advancing ODQA research and applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑