迈向高效教育聊天机器人：RAG框架基准评测
Towards Efficient Educational Chatbots: Benchmarking RAG Frameworks
AI总结：
本文提出一个基于生成式AI和RAG的GATE问答框架，通过基准评测最优嵌入模型与LLM配置，在延迟、忠实度和相关性上取得平衡，显著提升检索准确性与响应质量，为高效教育聊天机器人开发提供实用参考。
摘要：
大型语言模型（LLMs）通过获取大量基于文献的信息，在教育领域已被证明极具价值，使其能够在不依赖外部来源的情况下生成上下文。在本文中，我们提出了一个生成式AI驱动的GATE问答框架（GATE代表工程研究生能力倾向测试），该框架利用LLMs来解释GATE解答并支持学生的备考。我们进行了广泛的基准评测以选择最优的嵌入模型和LLM，基于延迟、忠实度和相关性等标准评估我们的框架，并通过人工评估进行额外验证。我们的聊天机器人集成了最先进的嵌入模型和LLMs，以提供准确、具备上下文感知能力的响应。通过严格的实验，我们确定了在性能和计算效率之间取得平衡的配置，确保聊天机器人能够可靠地满足学生需求。此外，我们讨论了在数据处理和建模中面临的挑战以及已实施的解决方案。我们的工作探索了检索增强生成（RAG）在GATE问答解释任务中的应用，研究结果表明检索准确性和响应质量有显著提升。本研究为开发有效的AI驱动教育工具提供了实用见解，同时指出了未来在可用性和可扩展性方面需要增强的领域。
英文摘要：
Large Language Models (LLMs) have proven immensely beneficial in education by capturing vast amounts of literature-based information, allowing them to generate context without relying on external sources. In this paper, we propose a generative AI-powered GATE question-answering framework (GATE stands for Graduate Aptitude Test in Engineering) that leverages LLMs to explain GATE solutions and support students in their exam preparation. We conducted extensive benchmarking to select the optimal embedding model and LLM, evaluating our framework based on criteria such as latency, faithfulness, and relevance, with additional validation through human evaluation. Our chatbot integrates state-of-the-art embedding models and LLMs to deliver accurate, context-aware responses. Through rigorous experimentation, we identified configurations that balance performance and computational efficiency, ensuring a reliable chatbot to serve students' needs. Additionally, we discuss the challenges faced in data processing and modeling and implemented solutions. Our work explores the application of Retrieval-Augmented Generation (RAG) for GATE Q/A explanation tasks, and our findings demonstrate significant improvements in retrieval accuracy and response quality. This research offers practical insights for developing effective AI-driven educational tools while highlighting areas for future enhancement in usability and scalability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
