arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.05600cs.CY
收藏

61A Bot报告:CS1课程中的AI助手节省学生作业时间并降低教职工需求(接下来该怎么办?)

61A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?)

J. D. Zamfirescu-Pereira, Laryn Qi, Björn Hartmann, John DeNero, Narges Norouzi

AI总结:

本文报告为大型CS1课程开发部署的GPT-4作业助手61A Bot,两学期超2000学生发起超10万请求,其在自动评分器中提供情境化反馈,结果显示作业时间缩短、论坛问题减少,并探讨了教学影响与成本收益。

摘要:

基于LLM的聊天机器人能让学生在作业中获得即时、交互式帮助,但即使是精心设计的机器人也可能无法满足所有教学目标。本文报告了为大型CS1课程学生开发并部署的基于GPT-4的交互式作业助手“61A Bot”;两个学期内,超2000名学生向该机器人发起超10万次请求。该助手在学生用于测试代码的命令行“自动评分器”中提供一次性、情境化反馈。机器人将学生代码包裹在自定义提示中,以支持教学目标并避免直接提供解决方案。通过分析学生反馈、问题和自动评分器数据,我们发现课程论坛中作业相关问题的发生率降低,且机器人可用时作业完成时间缩短:50-80百分位的学生每作业可节省超30分钟,比前学期同百分位学生少50%。最后,我们讨论了这些观察结果、对学生学习的潜在影响,以及CS1中AI辅助的其他潜在成本和收益。

英文摘要:

LLM-based chatbots enable students to get immediate, interactive help on homework assignments, but even a thoughtfully-designed bot may not serve all pedagogical goals. We report here on the development and deployment of a GPT-4-based interactive homework assistant ("61A Bot") for students in a large CS1 course; over 2000 students made over 100,000 requests of our Bot across two semesters. Our assistant offers one-shot, contextual feedback within the command-line "autograder" students use to test their code. Our Bot wraps student code in a custom prompt that supports our pedagogical goals and avoids providing solutions directly. Analyzing student feedback, questions, and autograder data, we find reductions in homework-related question rates in our course forum, as well as reductions in homework completion time when our Bot is available. For students in the 50th-80th percentile, reductions can exceed 30 minutes per assignment, up to 50% less time than students at the same percentile rank in prior semesters. Finally, we discuss these observations, potential impacts on student learning, and other potential costs and benefits of AI assistance in CS1.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑