arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.00072cs.IRcs.AIcs.CL
收藏

Pistis-RAG: 利用人类反馈增强检索增强生成

Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

Zhongguancun Laboratory(中关村实验室);Tsinghua University(清华大学);China Mobile Communications Group Co., Ltd.(中国移动通信集团有限公司)

Yu Bai, Yukai Miao, Li Chen, Dawei Wang, Dan Li, Yanyu Ren, Hongtao Xie, Ce Yang, Xuhui Cai

AI总结:

针对传统RAG仅靠语义相关性难保障生成质量、受提示排序影响的问题,研究提出以内容为核心的Pistis-RAG框架,利用人类反馈优化排序与生成,在MMLU和C-EVAL上准确率分别提升6.06%和7.08%。

摘要:

仅靠语义相关性无法保证生成质量提升时,RAG系统会面临局限性。由于大语言模型(LLM)对少样本提示的排序具有敏感性,这一问题会尤为突出,进而影响模型性能。为应对这一挑战,借助复制、重新生成或不喜欢这类结构化反馈让LLM输出与人类偏好对齐,是一种颇具前景的改进方法。这类反馈应用于整个输入列表,而非对单个文档给出具体评分,因此属于一种列表级标签的学习排序任务。\n 为解决该任务,我们提出Pistis-RAG——一种以内容为核心设计的新型RAG框架,旨在更好地让LLM与人类偏好对齐。Pistis-RAG能有效利用人类反馈,提升内容排序与生成质量。为验证我们的框架,我们使用公开数据集模拟人类反馈,从而有效评估并优化我们的方法。实验结果表明,相较于基线RAG系统,Pistis-RAG提升了与人类偏好的对齐度,在MMLU(英文)准确率指标上提升6.06%,在C-EVAL(中文)准确率指标上提升7.08%。这些结果凸显了Pistis-RAG在克服传统RAG方法相关局限性方面的有效性。

英文摘要:

RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. This issue becomes particularly evident due to the sensitivity of large language models (LLMs) to the ordering of few-shot prompts, which can affect model performance. To address this challenge, aligning LLM outputs with human preferences using structured feedback, such as options to copy, regenerate, or dislike, offers a promising method for improvement. This feedback is applied to the entire list of inputs rather than giving specific ratings for individual documents, making it a Listwide Labels Learning-to-Rank task. To address this task, we propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences. Pistis-RAG effectively utilizes human feedback, enhancing content ranking and generation quality. To validate our framework, we use public datasets to simulate human feedback, allowing us to evaluate and refine our method effectively. Experimental results indicate that Pistis-RAG improves alignment with human preferences relative to the baseline RAG system, showing a 6.06% increase in MMLU (English) and a 7.08% increase in C-EVAL (Chinese) accuracy metrics. These results highlight Pistis-RAG's effectiveness in overcoming the limitations associated with traditional RAG approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑