arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03708cs.CLcs.IR
收藏

用于上下文响应预测系统的基于 RAG 的问答

RAG based Question-Answering for Contextual Response Prediction System

The Home Depot(家得宝);The Pennsylvania State University(宾夕法尼亚州立大学)

Sriram Veturi, Saurabh Vaichal, Reshma Lal Jagadheesh, Nafis Irtiza Tripto, Nian Yan

AI总结:

本文提出端到端 RAG-LLM 框架,结合检索知识与聊天历史为零售客服生成建议,自动化与人工评估均显示其优于现有基于 BERT 的方法。

摘要:

大型语言模型(LLM)已在包括其作为高效问答系统潜力在内的各类自然语言处理(NLP)任务中展现出多功能性。然而,为了在行业场景中响应特定客户查询并提供精确且相关的信息,LLM 需要访问全面的知识库以避免幻觉。检索增强生成(RAG)成为应对这一挑战的一种有前景的技术。然而,使用 RAG 为真实应用开发准确的问答框架面临若干挑战:1)数据可用性问题;2)评估生成内容的质量;3)人工评估成本高昂。本文提出一种端到端框架,将具备 RAG 能力的 LLM 用于行业用例。给定客户查询,所提系统检索相关知识文档,并结合这些文档与先前聊天历史,为一家大型零售公司联络中心的客服人员生成响应建议。通过全面的自动化评估和人工评估,我们表明该解决方案在准确性和相关性方面优于现有基于 BERT 的算法。研究结果表明,基于 RAG 的 LLM 可通过减轻人工客服代表的工作负担,为其提供出色支持。

英文摘要:

Large Language Models (LLMs) have shown versatility in various Natural Language Processing (NLP) tasks, including their potential as effective question-answering systems. However, to provide precise and relevant information in response to specific customer queries in industry settings, LLMs require access to a comprehensive knowledge base to avoid hallucinations. Retrieval Augmented Generation (RAG) emerges as a promising technique to address this challenge. Yet, developing an accurate question-answering framework for real-world applications using RAG entails several challenges: 1) data availability issues, 2) evaluating the quality of generated content, and 3) the costly nature of human evaluation. In this paper, we introduce an end-to-end framework that employs LLMs with RAG capabilities for industry use cases. Given a customer query, the proposed system retrieves relevant knowledge documents and leverages them, along with previous chat history, to generate response suggestions for customer service agents in the contact centers of a major retail company. Through comprehensive automated and human evaluations, we show that this solution outperforms the current BERT-based algorithms in accuracy and relevance. Our findings suggest that RAG-based LLMs can be an excellent support to human customer service representatives by lightening their workload.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑