辅导对话中的模拟学生：实质还是幻象？
Simulated Students in Tutoring Dialogues: Substance or Illusion?
AI总结：
本文定义了学生模拟任务，提出涵盖语言、行为和认知的评估指标，并通过实验表明模拟学生的方法存在局限性，需进一步研究。
摘要：
大型语言模型（LLMs）的进步使教育领域出现了许多新的创新。然而，评估新技术的有效性需要真实的学生，这耗时且难以扩展。因此，许多近期基于LLM的辅导解决方案在训练和评估中都使用了模拟学生，通常通过简单的提示实现。令人惊讶的是，很少有工作致力于确保或甚至衡量模拟学生质量。在本工作中，我们正式定义了学生模拟任务，提出了一组涵盖语言、行为和认知方面的评估指标，并在这些指标上对广泛的模拟学生方法进行了基准测试。我们在一个真实世界的数学辅导对话数据集上进行了实验，其中自动和人工评估结果均显示，模拟学生的提示策略表现不佳；监督微调和偏好优化虽然表现更好但仍有局限性，这促使未来在这一具有挑战性的任务上进行更多研究。
英文摘要：
Advances in large language models (LLMs) enable many new innovations in education. However, evaluating the effectiveness of new technology requires real students, which is time-consuming and hard to scale up. Therefore, many recent works on LLM-powered tutoring solutions have used simulated students for both training and evaluation, often via simple prompting. Surprisingly, little work has been done to ensure or even measure the quality of simulated students. In this work, we formally define the student simulation task, propose a set of evaluation metrics that span linguistic, behavioral, and cognitive aspects, and benchmark a wide range of student simulation methods on these metrics. We experiment on a real-world math tutoring dialogue dataset, where both automated and human evaluation results show that prompting strategies for student simulation perform poorly; supervised fine-tuning and preference optimization yield much better but still limited performance, motivating future work on this challenging task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
