通过决策模型弥合新手与专家之间的差距：以纠正数学错误为例
Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes
AI总结：
本文提出 Bridge，通过认知任务分析将专家纠错思维转化为决策模型，并构建700段辅导对话数据集，显著提升 GPT4 的数学纠错辅导质量。
摘要：
规模化提供高质量辅导仍是教育领域的一项重大挑战。由于需求不断增长，许多平台聘用新手辅导教师；与经验丰富的教育工作者不同，他们难以处理学生的错误，因此无法抓住宝贵的学习机会。我们的工作探索了大型语言模型（LLM）在纠正数学错误方面弥合新手与专家知识差距的潜力。我们提出了 Bridge，这是一种利用认知任务分析将专家的潜在思维过程转化为用于纠错的决策模型的方法。该过程包括专家在生成回复之前识别（A）学生的错误、（B）补救策略以及（C）其意图。我们构建了一个包含 700 段真实辅导对话的数据集，并由专家标注其决策。我们在该数据集上评估了最先进的 LLM，发现专家决策模型对于 LLM 弥合差距至关重要：带有专家决策（例如“简化问题”）的 GPT4 回复比没有专家决策时的偏好度高 76%。此外，情境敏感的决策对于弥合教学差距至关重要：与专家决策相比，随机决策使 GPT4 的回复质量下降了 97%。我们的工作表明，将专家思维过程嵌入 LLM 生成过程，有望增强其弥合新手与专家知识差距的能力。我们的数据集和代码可在以下链接获取：https://github.com/rosewang2008/bridge。
英文摘要：
Scaling high-quality tutoring remains a major challenge in education. Due to growing demand, many platforms employ novice tutors who, unlike experienced educators, struggle to address student mistakes and thus fail to seize prime learning opportunities. Our work explores the potential of large language models (LLMs) to close the novice-expert knowledge gap in remediating math mistakes. We contribute Bridge, a method that uses cognitive task analysis to translate an expert's latent thought process into a decision-making model for remediation. This involves an expert identifying (A) the student's error, (B) a remediation strategy, and (C) their intention before generating a response. We construct a dataset of 700 real tutoring conversations, annotated by experts with their decisions. We evaluate state-of-the-art LLMs on our dataset and find that the expert's decision-making model is critical for LLMs to close the gap: responses from GPT4 with expert decisions (e.g., "simplify the problem") are +76% more preferred than without. Additionally, context-sensitive decisions are critical to closing pedagogical gaps: random decisions decrease GPT4's response quality by -97% than expert decisions. Our work shows the potential of embedding expert thought processes in LLM generations to enhance their capability to bridge novice-expert knowledge gaps. Our dataset and code can be found at: \url{https://github.com/rosewang2008/bridge}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
