arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.07559cs.CLcs.AIcs.LG
收藏

PaperQA:面向科研的检索增强生成式智能体

PaperQA: Retrieval-Augmented Generative Agent for Scientific Research

Future House(未来之家);Francis Crick Institute(弗朗西斯·克里克研究所);Align to Innovate(创新对齐组织);University of Oxford(牛津大学);University of Rochester(罗切斯特大学)

Jakub Lála, Odhran O'Donoghue, Aleksandar Shtedritski, Sam Cox, Samuel G. Rodriques, Andrew D. White

AI总结:

针对大语言模型的幻觉与不可解释性问题,提出面向科研的检索增强生成式智能体PaperQA,其性能优于现有同类模型,同时发布更复杂的文献问答基准LitQA,且该智能体在该基准上表现与人类专家相当。

摘要:

大语言模型(LLMs)在各类语言任务上泛化能力出色,但存在幻觉问题与不可解释性,导致若无真实基准则难以评估其准确性。学界已提出检索增强生成(RAG)模型以减少幻觉,并为答案的生成过程提供来源依据。将此类模型应用于科学文献领域,有望实现对科学知识的大规模、系统化处理。我们提出PaperQA——一款用于解答科学文献相关问题的RAG智能体。PaperQA作为智能体,可在全文科学文献中执行信息检索,评估来源与段落的相关性,并借助RAG生成答案。将该智能体作为问答模型测试后,我们发现其在现有科学问答基准上的表现优于现有LLM及LLM智能体。为推动该领域更贴近人类研究科学文献的方式,我们还推出了LitQA——一个更复杂的基准,要求从文献库中的全文科学论文里检索并综合信息。最后,我们验证了PaperQA在LitQA上的表现与人类专家研究者相当。

英文摘要:

Large Language Models (LLMs) generalize well across language tasks, but suffer from hallucinations and uninterpretability, making it difficult to assess their accuracy without ground-truth. Retrieval-Augmented Generation (RAG) models have been proposed to reduce hallucinations and provide provenance for how an answer was generated. Applying such models to the scientific literature may enable large-scale, systematic processing of scientific knowledge. We present PaperQA, a RAG agent for answering questions over the scientific literature. PaperQA is an agent that performs information retrieval across full-text scientific articles, assesses the relevance of sources and passages, and uses RAG to provide answers. Viewing this agent as a question answering model, we find it exceeds performance of existing LLMs and LLM agents on current science QA benchmarks. To push the field closer to how humans perform research on scientific literature, we also introduce LitQA, a more complex benchmark that requires retrieval and synthesis of information from full-text scientific papers across the literature. Finally, we demonstrate PaperQA's matches expert human researchers on LitQA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑