面向课堂的基于知识检索的LLM对齐——一项比较性RAG研究
Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Study
AI总结：
本研究比较向量与图检索增强生成在课堂问答中的效果，提出动态路由框架以平衡准确性与成本，为教育场景提供实用指导。
摘要：
像ChatGPT这样的大型语言模型在课堂上的使用日益增多，但它们常常提供过时或捏造的信息，这可能会误导学生。检索增强生成（RAG）通过将响应基于外部资源来提高LLM的可靠性。我们研究了两种可访问的RAG范式，即基于向量的检索和基于图的检索，以确定课堂问答（QA）的最佳实践。现有的比较研究未能考虑教学因素，如教育学科、问题类型和实际部署成本。使用一个新数据集EduScopeQA，包含跨学术科目的3,176个问题，我们衡量了从具体事实到广泛主题讨论的各种教育查询类型的性能。我们还使用一个系统性地修改过的教科书数据集来评估系统对齐性，这些教科书与LLM的潜在知识相矛盾。我们发现，OpenAI向量搜索RAG（代表基于向量的RAG）作为一个低成本的通才表现良好，特别是在快速事实检索方面。另一方面，GraphRAG Global在为主题查询提供教学丰富的答案方面表现出色，而GraphRAG Local在语料库完整性至关重要时，在密集的、修改过的教科书上实现了最高准确性。考虑到GraphRAG（代表基于图的RAG）的资源使用量高出10-20倍，我们展示了一个动态分支框架，将查询路由到最优检索方法，从而提高了保真度和效率。这些见解为教育工作者和系统设计者提供了可操作的指导，以便有效地将RAG增强的LLM集成到学习环境中。
英文摘要：
Large language models like ChatGPT are increasingly used in classrooms, but they often provide outdated or fabricated information that can mislead students. Retrieval Augmented Generation (RAG) improves reliability of LLMs by grounding responses in external resources. We investigate two accessible RAG paradigms, vector-based retrieval and graph-based retrieval to identify best practices for classroom question answering (QA). Existing comparative studies fail to account for pedagogical factors such as educational disciplines, question types, and practical deployment costs. Using a novel dataset, EduScopeQA, of 3,176 questions across academic subjects, we measure performance on various educational query types, from specific facts to broad thematic discussions. We also evaluate system alignment with a dataset of systematically altered textbooks that contradict the LLM's latent knowledge. We find that OpenAI Vector Search RAG (representing vector-based RAG) performs well as a low-cost generalist, especially for quick fact retrieval. On the other hand, GraphRAG Global excels at providing pedagogically rich answers to thematic queries, and GraphRAG Local achieves the highest accuracy with the dense, altered textbooks when corpus integrity is critical. Accounting for the 10-20x higher resource usage of GraphRAG (representing graph-based RAG), we show that a dynamic branching framework that routes queries to the optimal retrieval method boosts fidelity and efficiency. These insights provide actionable guidelines for educators and system designers to integrate RAG-augmented LLMs into learning environments effectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
