检索增强生成以改进数学问答：接地性与人类偏好之间的权衡
Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference
AI总结：
本文提出使用检索增强生成（RAG）结合开源数学教科书来改进中学数学问答，通过多条件调查发现人类偏好RAG回答，但需权衡回答的接地性与学生偏好。
摘要：
对于中学数学学生来说，与辅导老师进行交互式问答（QA）是一种有效的学习方式。生成式大语言模型（LLM）的灵活性和涌现能力引发了自动化辅导过程部分环节的兴趣激增——包括支持数学概念讨论的交互式问答。然而，LLM对数学问题的回答可能不正确或与教育情境不匹配——例如与学校的课程不一致。一个潜在的解决方案是检索增强生成（RAG），它涉及在LLM提示中纳入经过审核的外部知识源以提高回答质量。在本文中，我们设计了提示，检索并使用高质量开源数学教科书中的内容来生成对真实学生问题的回答。我们通过进行多条件调查来评估该RAG系统在中学代数和几何问答中的有效性，发现人类更喜欢使用RAG生成的回答，但当回答过于贴近教科书内容时则不然。我们认为，虽然RAG能够提高回答质量，但数学问答系统的设计者必须考虑生成学生偏好的回答与生成与特定教育资源紧密匹配的回答之间的权衡。
英文摘要：
For middle-school math students, interactive question-answering (QA) with tutors is an effective way to learn. The flexibility and emergent capabilities of generative large language models (LLMs) has led to a surge of interest in automating portions of the tutoring process - including interactive QA to support conceptual discussion of mathematical concepts. However, LLM responses to math questions can be incorrect or mismatched to the educational context - such as being misaligned with a school's curriculum. One potential solution is retrieval-augmented generation (RAG), which involves incorporating a vetted external knowledge source in the LLM prompt to increase response quality. In this paper, we designed prompts that retrieve and use content from a high-quality open-source math textbook to generate responses to real student questions. We evaluate the efficacy of this RAG system for middle-school algebra and geometry QA by administering a multi-condition survey, finding that humans prefer responses generated using RAG, but not when responses are too grounded in the textbook content. We argue that while RAG is able to improve response quality, designers of math QA systems must consider trade-offs between generating responses preferred by students and responses closely matched to specific educational resources.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
