Large Language Models (LLMs) have shown versatility in various Natural Language Processing (NLP) tasks, including their potential as effective question-answering systems. However, to provide precise and relevant information in response to specific customer queries in industry settings, LLMs require access to a comprehensive knowledge base to avoid hallucinations. Retrieval Augmented Generation (RAG) emerges as a promising technique to address this challenge. Yet, developing an accurate question-answering framework for real-world applications using RAG entails several challenges: 1) data availability issues, 2) evaluating the quality of generated content, and 3) the costly nature of human evaluation. In this paper, we introduce an end-to-end framework that employs LLMs with RAG capabilities for industry use cases. Given a customer query, the proposed system retrieves relevant knowledge documents and leverages them, along with previous chat history, to generate response suggestions for customer service agents in the contact centers of a major retail company. Through comprehensive automated and human evaluations, we show that this solution outperforms the current BERT-based algorithms in accuracy and relevance. Our findings suggest that RAG-based LLMs can be an excellent support to human customer service representatives by lightening their workload.
RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Xiaohongshu Inc.(小红书公司);Xi’an Jiaotong University(西安交通大学)
Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval-Augmented Generation (RAG)
1Department of Computer Science, Vanderbilt University, Nashville, USA 2College of Engineering \& Science, University of Detroit Mercy, Detroit, USA 3The School for Science;Math, Vanderbilt University, Nashville, USA
PACE-RAG: Patient-Aware Contextual and Evidence-Constrained RAG for Clinical Drug Recommendation
Korea Advanced Institute of Science and Technology(韩国科学技术院);Department of Neurology, Seoul National University Hospital(首尔国立大学医院神经科);Asan Medical Center, University of Ulsan, College of Medicine(釜山大学医学院阿桑医院);Haeundae Paik Hospital, Inje University(庆尚大学医院海undae医院)
Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaboration
Emory University(埃默里大学);University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心);Georgia Institute of Technology(佐治亚理工学院);SUNY Albany(纽约州立大学奥尔巴尼分校)
REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);National University of Singapore(新加坡国立大学);Baidu Inc.(百度公司);School of Information, Renmin University of China(中国人民大学信息学院)
DocNavRAG: Document-Structured Graph RAG with Stateful Evidence Construction for Complex Document Question Answering
School of Computer Science, Wuhan University(武汉大学计算机学院);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong(香港中文大学);ETH Zurich(苏黎世联邦理工学院)
University of Macau(澳门大学);Xiamen University(厦门大学);Peking University(北京大学);Hanyang University(翰阳大学);University of Liverpool(利物浦大学);Zhejiang University(浙江大学)
Exploring the Role of Knowledge Graph-Based RAG in Japanese Medical Question Answering with Small-Scale LLMs
The University of Tokyo(东京大学);NEC Laboratories America(NEC美国实验室);Dokkyo Medical University(多摩医学大学);Graduate School of Frontier Sciences, The University of Tokyo(东京大学前沿科学研究生院)
First Token Probability Guided RAG for Telecom Question Answering
Shenzhen Research Institute of Big Data(深圳大数据研究院);School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳)理工学院);School of Computer Science and Engineering, Sun Yat-sen University(中山大学计算机科学与工程学院)
Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoring
Faculty of Education, The University of Hong Kong(香港大学教育学院);Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology(香港科学与技术大学土木与环境工程系)
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering
MoE Key Lab of Artificial Intelligence(摩尔电子关键实验室);X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University(X-LANCE实验室,计算机科学学院,上海交通大学);Jiangsu Key Lab of Language Computing(江苏语言计算重点实验室);AISpeech Co., Ltd.(AISpeech公司);Suzhou Laboratory(苏州实验室)