arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.01158cs.CL
收藏

学习探索与选择:面向覆盖条件检索增强生成

Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);LG AI Research(LG AI研究院);Cornell University(康奈尔大学);Korea University(高丽大学);University of Illinois Chicago(芝加哥伊利诺伊大学)

Takyoung Kim, Kyungjae Lee, Young Rok Jang, Ji Yong Cho, Gangwoo Kim, Minseok Cho, Moontae Lee

AI总结:

针对覆盖条件查询下LLM响应冗长且不贴合需求的问题,提出QTree数据集与QPlanner模型,通过生成查询大纲引导RAG系统,实验证明其能生成更高质量大纲以更好满足用户需求。

摘要:

与大型语言模型(LLM)的交互通常会产生冗长且详细的响应,同时利用参数化知识和检索增强生成(RAG)。虽然这些响应可以提供丰富的见解,但它们往往包含冗余或不太吸引人的内容,且不符合用户兴趣。当用户指定要包含或排除的特定子主题(称为覆盖条件(C²)查询)时,这一问题变得明显,因为LLM往往难以提供量身定制的响应。为了解决这一挑战,我们研究了查询大纲的作用,即一系列子查询,旨在引导LLM生成满足特定用户要求的响应。为了系统地创建和评估这些大纲,我们引入了QTree,一个包含10K个分层信息寻求子查询集的数据集,这些子查询在C²场景中为大纲创建和评估定义了结构化边界。此外,我们开发了QPlanner,一个7B语言模型,训练用于在QTree边界内生成定制大纲。我们通过自动和人工判断评估生成大纲的有效性,重点关注它们在检索增强生成(RAG)系统中的影响。实验结果表明,QPlanner,尤其是在使用DPO等对齐技术训练时,能生成更高质量的大纲,更好地满足多样化的用户需求。

英文摘要:

Interactions with large language models (LLMs) often yield long and detailed responses, leveraging both parametric knowledge and retrieval-augmented generation (RAG). While these responses can provide rich insights, they often include redundant or less engaging content not aligned with user interests. This issue becomes apparent when users specify particular subtopics to include or exclude -- termed coverage-conditioned ($C^2$) queries -- as LLMs often struggle to provide tailored responses. To address this challenge, we investigate the role of query outlines, sequences of subqueries designed to guide LLMs in generating responses that meet specific user requirements. To systematically create and evaluate these outlines, we introduce QTree, a dataset of 10K hierarchical sets of information-seeking subqueries that define structured boundaries for outline creation and evaluation in $C^2$ scenarios. Additionally, we develop QPlanner, a 7B language model trained to generate customized outlines within boundaries of QTree. We evaluate the effectiveness of the generated outlines through automatic and human judgements, focusing on their impact within retrieval-augmented generation (RAG) systems. Experimental results demonstrate that QPlanner, especially when trained with alignment techniques like DPO, generates higher-quality outlines that better fulfill diverse user needs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑