大型语言模型作为思考出声的学生：过于连贯、啰嗦且自信
Large Language Models as Students Who Think Aloud: Overly Coherent, Verbose, and Confident
AI总结：
本文评估LLM在模拟学生思考过程中的表现，发现其推理过于连贯、啰嗦且缺乏变异性，揭示了使用生成式AI设计适应性系统时的认知局限。
摘要：
大型语言模型（LLMs）正越来越多地融入基于AI的辅导系统中。它们能否忠实建模初学者的推理和元认知判断？现有评估侧重于问题解决准确性，忽略了人类学习中碎片化和不完美推理的特点。我们通过630个来自多步骤化学辅导问题的think-aloud陈述，结合学生提示使用、尝试和问题上下文的日志，评估LLM作为初学者的表现。在最小和扩展上下文提示下，我们将LLM生成的推理与人类学习者的陈述进行比较，并评估模型预测步骤级学习者成功的能力。尽管GPT-4.1生成流畅且上下文合适的延续，但其推理系统性地过于连贯、啰嗦且比人类think-alouds更不具有变异性。这些影响在提示过程中问题解决上下文更丰富时加剧。学习者表现被一致高估。这些发现突显了用LLM模拟学习的认知局限。我们将其限制归因于LLM训练数据，包括缺乏情感表达和问题解决过程中工作记忆限制的专家似解决方案。我们的评估框架可以指导未来设计更忠实支持初学者学习和自我调节的适应性系统。
英文摘要：
Large language models (LLMs) are increasingly embedded in AI-based tutoring systems. Can they faithfully model novice reasoning and metacognitive judgments? Existing evaluations emphasize problem-solving accuracy, overlooking the fragmented and imperfect reasoning that characterizes human learning. We evaluate LLMs as novices using 630 think-aloud utterances from multi-step chemistry tutoring problems with problem-solving logs of student hint use, attempts, and problem context. We compare LLM-generated reasoning to human learner utterances under minimal and extended contextual prompting, and assess the models' ability to predict step-level learner success. Although GPT-4.1 generates fluent and contextually appropriate continuations, its reasoning is systematically over-coherent, verbose, and less variable than human think-alouds. These effects intensify with a richer problem-solving context during prompting. Learner performance was consistently overestimated. These findings highlight epistemic limitations of simulating learning with LLMs. We attribute these limitations to LLM training data, including expert-like solutions devoid of expressions of affect and working memory constraints during problem solving. Our evaluation framework can guide future design of adaptive systems that more faithfully support novice learning and self-regulation using generative artificial intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
