监督微调大语言模型使其在编程教育中充当教学代理
Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education
AI总结：
本文提出通过监督微调大语言模型（GuideLM）提升编程教育中的教学效果，集成于DCC编译器，实验显示苏格拉底式引导提升8%、用词经济性提升58%，但准确性略有下降。
摘要：
大语言模型（LLMs）在高等教育中的应用日益增多，但其作为教学代理的有效性仍未得到充分检验。在本文中，我们介绍了GuideLM的开发过程，这是一个专为编程教育设计的微调大语言模型。GuideLM已被集成到调试C编译器（DCC）中，这是一个利用大语言模型生成具有教学意义的错误解释的教育型C编译器。此前，DCC依赖现成的OpenAI模型，这些模型虽然准确，但常常过度帮助学生，尽管有相反的提示，仍直接提供解决方案。为了解决这一问题，我们对528个学生问题/教师回答配对的数据集采用了监督微调（SFT），创建了两个模型：GuideLM和GuideLM-mini，分别基于ChatGPT-4o和4o-mini进行微调。我们对每个模型的400个回答进行了专家分析，将其教学效果与基础OpenAI模型进行比较。我们的评估基于建构主义和认知负荷理论，评估了概念支架、清晰度和苏格拉底式引导等因素。结果表明，GuideLM和GuideLM-mini提高了教学表现，与GPT-4o相比，苏格拉底式引导增加了8%，用词经济性提高了58%。然而，这种改进是以通用准确性略有下降为代价的。虽然还需要进一步的工作，但我们的研究结果表明，使用针对性数据集微调大语言模型是开发更适合教育场景的模型的一种有前景的方法。
英文摘要：
Large language models (LLMs) are increasingly being explored in higher education, yet their effectiveness as teaching agents remains underexamined. In this paper, we present the development of GuideLM, a fine-tuned LLM designed for programming education. GuideLM has been integrated into the Debugging C Compiler (DCC), an educational C compiler that leverages LLMs to generate pedagogically sound error explanations. Previously, DCC relied on off-the-shelf OpenAI models, which, while accurate, often over-assisted students by directly providing solutions despite contrary prompting. To address this, we employed supervised fine-tuning (SFT) on a dataset of 528 student-question/teacher-answer pairs, creating two models: GuideLM and GuideLM-mini, fine-tuned on ChatGPT-4o and 4o-mini, respectively. We conducted an expert analysis of 400 responses per model, comparing their pedagogical effectiveness against base OpenAI models. Our evaluation, grounded in constructivism and cognitive load theory, assessed factors such as conceptual scaffolding, clarity, and Socratic guidance. Results indicate that GuideLM and GuideLM-mini improve pedagogical performance, with an 8% increase in Socratic guidance and a 58% improvement in economy of words compared to GPT-4o. However, this refinement comes at the cost of a slight reduction in general accuracy. While further work is needed, our findings suggest that fine-tuning LLMs with targeted datasets is a promising approach for developing models better suited to educational contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
