明辨之心还是通用导师？评估苏格拉底式 LLM 的教学引导能力
Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs
AI总结：
本文提出基于真实教育对话的 GuideEval 基准，从感知、统筹和启发三阶段评估苏格拉底式 LLM 的自适应教学引导能力，并通过行为引导微调显著改善其表现。
摘要：
大型语言模型的对话能力为实现可扩展的交互式辅导带来了巨大前景。尽管以往研究主要考察了它们生成苏格拉底式问题的能力，却常常忽视一个关键方面：根据学习者的认知状态自适应地引导学习者。本研究超越问题生成，强调教学引导能力。我们提出问题：LLM 能否模仿专家导师，根据学习者的状态动态调整策略？为研究这一点，我们提出 GuideEval，这是一个基于真实教育对话的基准，通过三阶段行为框架评估教学引导：（1）感知，推断学习者状态；（2）统筹，调整教学策略；（3）启发，激发恰当反思。实证结果表明，当学习者感到困惑或需要重新引导时，现有 LLM 往往无法提供有效的自适应支架。为补充定量评估，我们进行了详细的失败案例分析，以直观理解这些不足。此外，我们引入一种行为引导的微调策略，利用由行为提示的教学对话，显著提升引导性能。通过将关注点从孤立的内容评估转向以学习者为中心、具备状态感知的交互，我们的工作倡导采用更具对话性的范式来评估苏格拉底式 LLM。
英文摘要：
The conversational capabilities of large language models hold significant promise for enabling scalable and interactive tutoring. While prior research has primarily examined their ability to generate Socratic questions, it often overlooks a critical aspect: adaptively guiding learners in accordance with their cognitive states. This study moves beyond question generation to emphasize instructional guidance capability. We ask: Can LLMs emulate expert tutors who dynamically adjust strategies in response to learners' states? To investigate this, we propose GuideEval, a benchmark grounded in authentic educational dialogues that evaluates pedagogical guidance through a three-phase behavioral framework: (1) Perception, inferring learner states; (2) Orchestration, adapting instructional strategies; and (3) Elicitation, stimulating proper reflections. Empirical results indicate that existing LLMs often fail to provide effective adaptive scaffolding when learners experience confusion or require redirection. To complement the quantitative evaluation, we conduct a detailed failure case analysis, providing an intuitive understanding of these shortcomings. Furthermore, we introduce a behavior-guided finetuning strategy that leverages behavior-prompted instructional dialogues, substantially enhancing guidance performance. By shifting the focus from isolated content evaluation to learner-centered state-aware interaction, our work advocates a more dialogic paradigm for evaluating Socratic LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
