大型语言模型在数学辅导中接近专家教学质量,但在教学和语言特征上存在差异
Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles
AI总结:
通过分析数学辅导对话数据集,比较专家、新手教师和七种大型语言模型的教学质量,发现大型语言模型平均接近专家水平,但在教学策略和语言特征上存在系统性差异。
摘要:
最近的工作探索了使用大型语言模型(LLMs)生成数学辅导回应,但尚不清楚其教学行为与人类专家实践的接近程度。我们分析了一个数学补救对话数据集,其中专家教师、新手教师和七种不同规模的大型语言模型(包括开放权重和商业模型)对相同的学生错误做出回应。我们检查了教学策略和辅导回应的语言特征,包括吸收(重述和转述)、追问准确性和推理、词汇多样性、可读性、礼貌性和能动性。我们发现专家教师产生的回应质量高于新手教师,并且较大的LLMs通常比较小的模型获得更高的教学质量评分,平均接近专家表现。然而,LLMs在教学特征上表现出系统性差异:它们较少使用专家教师特有的讨论策略,同时生成更长、词汇更丰富、更礼貌的回应。回归分析表明,追问准确性和推理、重述和转述以及词汇多样性与感知教学质量正相关,而更高水平的能动性和礼貌性语言则负相关。这些发现强调了在评估人类教师和智能辅导系统的辅导回应时分析教学策略和语言特征的重要性。
英文摘要:
Recent work has explored the use of large language models (LLMs) to generate tutoring responses in mathematics, yet it remains unclear how closely their instructional behavior aligns with expert human practice. We analyze a dataset of math remediation dialogues in which expert tutors, novice tutors, and seven LLMs of varying sizes, comprising both open-weight and commercial models, respond to the same student errors. We examine instructional strategies and linguistic characteristics of tutoring responses, including uptake (restating and revoicing), pressing for accuracy and reasoning, lexical diversity, readability, politeness, and agency. We find that expert tutors produce higher-quality responses than novices, and that larger LLMs generally receive higher pedagogical quality ratings than smaller models, approaching expert performance on average. However, LLMs exhibit systematic differences in their instructional profiles: they underuse discursive strategies characteristic of expert tutors while generating longer, more lexically diverse, and more polite responses. Regression analyses show that pressing for accuracy and reasoning, restating and revoicing, and lexical diversity, are positively associated with perceived pedagogical quality, whereas higher levels of agentic and polite language are negatively associated. These findings highlight the importance of analyzing instructional strategies and linguistic characteristics when evaluating tutoring responses across human tutors and intelligent tutoring systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。