利用大语言模型探索师生对话中的知识追踪
Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs
AI总结：
本研究针对开放式对话辅导中知识追踪难的问题，提出用LLM识别对话知识组件与学生回应正确性，结合LLMKT方法，在两个数据集上实现了比现有KT方法更优的学生回应正确性预测效果。
摘要：
大语言模型（LLM）的最新进展推动了人工智能（AI）驱动的辅导聊天机器人的发展，其在提供广泛的高质量个性化教育渠道方面展现出应用前景。现有研究已探讨如何让LLM遵循教学原则，但尚未研究LLM在支持辅导方面的更广泛用途。截至目前，在开放式对话辅导中实现学生知识追踪与错误概念分析，既困难又耗时。本研究探究了LLM是否能为该任务提供支持：我们首先采用LLM提示方法识别每轮对话（即辅导者提出任务的话语或学生回应的话语）中涉及的知识组件/技能。我们还评估学生对辅导者的回应是否正确，并通过人类专家标注验证LLM的准确率。随后，我们在得到的标注数据上应用一系列知识追踪（KT）方法，以追踪整个对话过程中的学生知识水平。我们在两个辅导对话数据集上开展实验，结果表明一种新颖且简单的基于LLM的方法LLMKT，在预测对话中学生回应正确性方面显著优于现有KT方法。我们开展了广泛的定性分析，明确了对话知识追踪（dialogueKT）中的挑战，并提出了多个未来研究方向。
英文摘要：
Recent advances in large language models (LLMs) have led to the development of artificial intelligence (AI)-powered tutoring chatbots, showing promise in providing broad access to high-quality personalized education. Existing works have studied how to make LLMs follow tutoring principles, but have not studied broader uses of LLMs for supporting tutoring. Up until now, tracing student knowledge and analyzing misconceptions has been difficult and time-consuming to implement for open-ended dialogue tutoring. In this work, we investigate whether LLMs can be supportive of this task: we first use LLM prompting methods to identify the knowledge components/skills involved in each dialogue turn, i.e., a tutor utterance posing a task or a student utterance that responds to it. We also evaluate whether the student responds correctly to the tutor and verify the LLM's accuracy using human expert annotations. We then apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue. We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues. We perform extensive qualitative analyses to highlight the challenges in dialogueKT and outline multiple avenues for future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
