arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.06384cs.CLcs.AI
收藏

迈向可靠且流畅的大型语言模型:在QA系统中引入反馈学习循环

Towards Reliable and Fluent Large Language Models: Incorporating Feedback Learning Loops in QA Systems

Naver Corp.(Naver公司);Korea University(高丽大学)

Dongyub Lee, Taesun Whang, Chanhee Lee, Heuiseok Lim

AI总结:

本研究构建数据集训练批评模型评估LLM在QA系统中生成响应的引用、正确性和流畅性,并提出自动化反馈机制与反馈学习循环迭代优化模型,使ChatGPT引用精度提升4%、流畅性提升约8%且保持高正确性。

摘要:

大型语言模型(LLMs)已成为各种日常应用中的多功能工具。然而,它们存在一些问题,削弱了其实用性和可信度。这些问题包括引入错误引用(citation)、生成幻觉信息(correctness)以及包含多余或遗漏关键细节(fluency)。为了改善这些问题,本研究做出了几项关键贡献。首先,我们构建了一个数据集,用于训练一个能够评估QA系统中LLMs生成响应的引用、正确性和流畅性的批评模型。其次,我们提出了一种自动化反馈机制,利用该批评模型对生成文本的异构方面提供实时反馈。第三,我们引入了一个反馈学习循环,使用该批评模型迭代地提高负责响应生成的LLM的性能。实验结果证明了我们方法的有效性,显示ChatGPT在引用和流畅性指标上有显著提升,包括引用精度提高4%,流畅性MAUVE指标提升约8%,同时保持高水平的正确性。

英文摘要:

Large language models (LLMs) have emerged as versatile tools in various daily applications. However, they are fraught with issues that undermine their utility and trustworthiness. These include the incorporation of erroneous references (citation), the generation of hallucinated information (correctness), and the inclusion of superfluous or omission of crucial details (fluency). To ameliorate these concerns, this study makes several key contributions. First, we build a dataset to train a critic model capable of evaluating the citation, correctness, and fluency of responses generated by LLMs in QA systems. Second, we propose an automated feedback mechanism that leverages the critic model to offer real-time feedback on heterogeneous aspects of generated text. Third, we introduce a feedback learning loop that uses this critic model to iteratively improve the performance of the LLM responsible for response generation. Experimental results demonstrate the efficacy of our approach, showing substantial improvements in citation and fluency metrics for ChatGPT, including a 4% precision increase in citation and an approximately 8% enhancement in the MAUVE metric for fluency, while maintaining high levels of correctness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑