MathDial：一个基于数学推理问题且具有丰富教学属性的对话辅导数据集
MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
AI总结：
提出人机协作框架构建MathDial数据集，通过教师引导LLM模拟学生错误，生成3k个基于数学推理的辅导对话，并利用其微调出能平衡解题与教学的更优导师模型。
摘要：
尽管自动对话导师在实现教育个性化和提高可及性方面具有巨大潜力，但此类系统的研究一直受限于缺乏足够大且高质量的数据集。收集此类数据集仍然充满挑战，因为记录辅导过程会引发隐私担忧，而众包则会导致数据质量不足。为了解决这一问题，我们提出了一个框架，通过将人类教师与一个被提示代表常见学生错误的大型语言模型（LLM）配对来生成此类对话。我们描述了如何使用该框架收集 MathDial，这是一个包含 3k 个基于多步数学推理问题的一对一师生辅导对话的数据集。虽然像 GPT-3 这样的模型是很好的问题解决者，但它们在辅导方面却失败了，因为它们会生成事实上不正确的反馈，或者倾向于过早向学生透露解决方案。为了克服这一点，我们让教师根据教师行为分类法，使用各种脚手架问题引导学生，从而为学生提供学习机会。我们证明了 MathDial 及其丰富的注释可用于微调模型，使其成为更有效的导师（而不仅仅是解决者）。我们通过自动和人工评估证实了这一点，特别是在一个交互式环境中，该环境衡量了学生解决成功与告知解决方案之间的权衡。该数据集已公开发布。
英文摘要：
While automatic dialogue tutors hold great potential in making education personalized and more accessible, research on such systems has been hampered by a lack of sufficiently large and high-quality datasets. Collecting such datasets remains challenging, as recording tutoring sessions raises privacy concerns and crowdsourcing leads to insufficient data quality. To address this, we propose a framework to generate such dialogues by pairing human teachers with a Large Language Model (LLM) prompted to represent common student errors. We describe how we use this framework to collect MathDial, a dataset of 3k one-to-one teacher-student tutoring dialogues grounded in multi-step math reasoning problems. While models like GPT-3 are good problem solvers, they fail at tutoring because they generate factually incorrect feedback or are prone to revealing solutions to students too early. To overcome this, we let teachers provide learning opportunities to students by guiding them using various scaffolding questions according to a taxonomy of teacher moves. We demonstrate MathDial and its extensive annotations can be used to finetune models to be more effective tutors (and not just solvers). We confirm this by automatic and human evaluation, notably in an interactive setting that measures the trade-off between student solving success and telling solutions. The dataset is released publicly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
