学生不会像你希望的那样使用LLMs
Your Students Don't Use LLMs Like You Wish They Did
AI总结：
本文提出六个计算指标，用于评估学生与AI对话的教育对齐情况，发现部署环境是影响使用模式的主要因素，而非学生偏好或系统设计。
摘要：
教育NLP系统通常通过参与度指标和满意度调查来评估，这些指标在最佳情况下只是实现教学目标的代理。我们引入了六个计算指标，用于自动化评估学生-AI对话中的教学对齐情况。我们通过分析来自四个课程的500次对话中的12,650条信息，验证了我们的指标。使用我们的指标，我们发现了一个根本性的不匹配：教育者为持续学习对话设计对话辅导系统，但学生主要用它们来提取答案。部署环境是使用模式的最强预测因素，超过学生偏好或系统设计：当AI工具是可选的时，使用集中在截止日期附近；当整合到课程结构中时，学生会要求解决verbatim作业问题。全对话评估遗漏了这些逐轮模式。我们的指标将使研究人员在构建教育对话系统时能够衡量是否实现了教学目标。
英文摘要：
Educational NLP systems are typically evaluated using engagement metrics and satisfaction surveys, which are at best a proxy for meeting pedagogical goals. We introduce six computational metrics for automated evaluation of pedagogical alignment in student-AI dialogue. We validate our metrics through analysis of 12,650 messages across 500 conversations from four courses. Using our metrics, we identify a fundamental misalignment: educators design conversational tutors for sustained learning dialogue, but students mainly use them for answer-extraction. Deployment context is the strongest predictor of usage patterns, outweighing student preference or system design: when AI tools are optional, usage concentrates around deadlines; when integrated into course structure, students ask for solutions to verbatim assignment questions. Whole-dialogue evaluation misses these turn-by-turn patterns. Our metrics will enable researchers building educational dialogue systems to measure whether they are achieving their pedagogical goals.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
