教育技术中利用大型语言模型自动生成数学问题提示
Automatic Generation of Question Hints for Mathematics Problems using Large Language Models in Educational Technology
AI总结：
本研究利用大型语言模型作为教师自动生成数学问题提示，发现针对特定错误和基于常见错误的提示最有效，且Llama-3-8B-Instruct作为教师整体优于GPT-4o，同时提示能显著提升模拟学生的自我纠正能力。
摘要：
在智能辅导系统（ITSs）中，由大型语言模型（LLMs）自动生成提示已显示出增强学生学习的潜力。然而，生成既符合教学法要求、又能解决学生误解并遵循特定教育目标的提示仍然具有挑战性。本研究探索使用LLMs（GPT-4o和Llama-3-8B-instruct）作为教师，为通过LLMs（GPT-3.5-turbo、Llama-3-8B-Instruct或Mistral-7B-instruct-v0.3）模拟的学生生成有效提示，这些学生解答为人类高中生设计、并基于认知科学原理设计的数学练习。我们在此展示对以下几个维度的研究：1）识别模拟学生在中学数学练习中犯的错误模式；2）为作为教师的GPT-4o开发各种提示词，并评估其在生成使模拟学生能够自我纠正的提示方面的有效性；3）基于生成相关提示和促进错误纠正的能力，测试表现最佳的提示词，并使用Llama-3-8B-Instruct作为教师，以便与GPT-4o进行性能比较。结果表明，模型错误随着温度设置的升高而增加。值得注意的是，当提示由GPT-4o生成时，最有效的提示包括针对特定错误定制的提示以及基于常见数学错误提供通用提示的提示。有趣的是，作为教师的Llama-3-8B-Instruct在整体表现上优于GPT-4o。此外，作为学生的LLMs，特别是GPT-3.5-turbo，在收到提示后，其问题解决和答案修订能力显著提高，尤其是在较低温度设置下。然而，像Mistral-7B-Instruct这样的模型在温度升高时表现出性能下降。
英文摘要：
The automatic generation of hints by Large Language Models (LLMs) within Intelligent Tutoring Systems (ITSs) has shown potential to enhance student learning. However, generating pedagogically sound hints that address student misconceptions and adhere to specific educational objectives remains challenging. This work explores using LLMs (GPT-4o and Llama-3-8B-instruct) as teachers to generate effective hints for students simulated through LLMs (GPT-3.5-turbo, Llama-3-8B-Instruct, or Mistral-7B-instruct-v0.3) tackling math exercises designed for human high-school students, and designed using cognitive science principles. We present here the study of several dimensions: 1) identifying error patterns made by simulated students on secondary-level math exercises; 2) developing various prompts for GPT-4o as a teacher and evaluating their effectiveness in generating hints that enable simulated students to self-correct; and 3) testing the best-performing prompts, based on their ability to produce relevant hints and facilitate error correction, with Llama-3-8B-Instruct as the teacher, allowing for a performance comparison with GPT-4o. The results show that model errors increase with higher temperature settings. Notably, when hints are generated by GPT-4o, the most effective prompts include prompts tailored to specific errors as well as prompts providing general hints based on common mathematical errors. Interestingly, Llama-3-8B-Instruct as a teacher showed better overall performance than GPT-4o. Also the problem-solving and response revision capabilities of the LLMs as students, particularly GPT-3.5-turbo, improved significantly after receiving hints, especially at lower temperature settings. However, models like Mistral-7B-Instruct demonstrated a decline in performance as the temperature increased.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
