arXiv 2507.174422025-11-30cs.CLcs.AI

Confident RAG: 通过多嵌入和置信度评分提升LLM在数学问题回答中的性能

Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoring

Faculty of Education, The University of Hong Kong（香港大学教育学院）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学土木与环境工程系）

Shiting Chen, Zijian Zhao, Jinsong Chen

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

Confident RAG通过多嵌入和置信度评分提升LLM在数学问题回答中的性能，实验表明其在准确率上比普通LLMs和RAG分别提升10%和5%。

摘要：

大型语言模型（LLMs）在数学教育中具有重要潜力，但它们常常在复杂的数学推理上遇到困难。虽然检索增强生成（RAG）通过将LLMs与外部知识相结合来缓解这些问题，但其效果仍然不稳定，严重依赖于单一嵌入模型的选择。在静态RAG工作流之外，我们借鉴了代理工作流模式，这是一种引入结构化任务分解和协作以增强系统性能的范式。我们提出并检验了两种新的方法，结合多种嵌入模型的优势。虽然我们的混合嵌入RAG方法（融合检索文档）显示出有限的提升，但我们的Confident RAG方法（生成多个答案并选择置信度最高的答案）显示出显著的改进。实验结果表明，Confident RAG在普通LLMs上平均准确率提高了约10%，在普通RAG上提高了5%。在不同LLM和嵌入模型之间的一致结果表明，Confident RAG是高效且易于使用的可信数学AI助手解决方案。最后，我们讨论了这项工作如何为教育环境中部署代理RAG系统奠定基础，其中自主规划和迭代完善可以在我们的稳健检索基础上进行构建。

英文摘要：

Large Language Models (LLMs) hold significant promise for mathematics education, yet they often struggle with complex mathematical reasoning. While Retrieval-Augmented Generation (RAG) mitigates these issues by grounding LLMs in external knowledge, its effectiveness remains unstable, heavily dependent on the choice of a single embedding model. Moving beyond static RAG workflows, we draw on agentic workflow patterns, a paradigm that introduces structured task decomposition and collaboration to enhance system performance. We propose and examine two novel approaches that combine the benefits of multiple embedding models. While our Mixture-Embedding RAG approach (fusing retrieved documents) shows limited gains, our Confident RAG method (generating multiple answers and selecting the one with the highest confidence score) demonstrates significant improvement. Experimental results show that Confident RAG achieved average accuracy improvements of approximately 10% over vanilla LLMs and 5% over vanilla RAG. The consistent results across different LLMs and embedding models indicate that Confident RAG is an efficient plug-and-play solution for trustworthy mathematical AI assistants. Finally, we discuss how this work lays the groundwork for deploying Agentic RAG systems in educational settings, where autonomous planning and iterative refinement can be built upon our robust retrieval foundation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。