护栏还是路障？教学风格与上下文感知在编程AI助教中的影响

中文摘要 AI 辅助

由大型语言模型（LLMs）和教学护栏支持的AI助教（AI TAs）正越来越多地融入编程课程，为学生提供可扩展的提示、概念解释和代码级反馈。然而，护栏也可能产生摩擦。如果学生觉得所提供的支持过于限制性或与当前进度缺乏良好上下文关联，他们可能会绕过经批准的工具，转而使用通用LLM。为了调查AI助教设计如何影响学生的学习体验，我们在一个入门编程课程中对132名学生进行了随机对照试验。学生完成了三项与代码编写和调试相关的任务，并被随机分配到四种AI助教之一，这些AI助教在两个维度上有所不同：教学指导风格（苏格拉底式vs.直接指导）和上下文感知（无上下文vs.包含问题和学生解决方案的完整上下文）。我们考察了学生的感知、互动行为以及任务后理解的证据。学生对具有完整上下文的苏格拉底式AI助教评价最低，报告了对任务完成的感知支持显著较低。描述性地，该条件也显示出最高的观察互动压力、最高的外部LLM使用率以及最低的展示完全理解的任务后解释比例，尽管这些差异在统计上不显著。这些发现表明，有护栏的AI助教并不自动地更有利于学习。相反，它们的有效性取决于教学指导和上下文感知如何以学生认为有用、支持且值得继续使用的方式取得平衡。

英文摘要

AI teaching assistants (AI TAs) backed by large language models (LLMs) and pedagogical guardrails are increasingly being integrated into programming courses, providing students with scalable access to hints, conceptual explanations, and code-level feedback. However, guardrails may also create friction. If students feel that the support provided is overly restrictive or poorly contextualized to their current progress, they may bypass approved tools for general-purpose LLMs. To investigate how AI TA design affects students' learning experiences, we conducted a randomized controlled trial with 132 students in an introductory programming course. Students completed three tasks related to code-writing and debugging and were randomly assigned to one of four AI TAs varied across two dimensions: pedagogical guidance style (Socratic vs. Direct instruction) and context awareness (no context vs. full context of the problem and student solution). We examined students' perceptions, interaction behaviors, and evidence of post-task comprehension. Students rated the Socratic AI TA with full context least favorably, reporting significantly lower perceived support for task completion. Descriptively, this condition also showed the highest observed interaction stress, the highest rate of external LLM use, and the lowest proportion of post-task explanations demonstrating full comprehension, though these differences were not statistically significant. These findings suggest that guardrailed AI TAs are not automatically better for learning. Instead, their effectiveness depends on how pedagogical guidance and contextual awareness are balanced in ways that students experience as useful, supportive, and worth continuing to use.

发表机构

• University of Florida（佛罗里达大学）
• University of Auckland（奥克兰大学）

机构由 AI 辅助整理，请以论文原文为准。