arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.05068cs.SEcs.CY
收藏

当人工智能故意出错时:学生如何应对有缺陷的生成式人工智能代码

When AI Is Wrong on Purpose: How Students Respond to Buggy GenAI Code

MPI-SWS(马克斯·普朗克软件系统研究所);University of Auckland(奥克兰大学);Aalto University(阿尔托大学);Abilene Christian University(阿比林基督教大学)

Victor-Alexandru Pădurean, Kaitlin Riegel, Alkis Gotovos, Jyotika Mahapatra, Ahana Ghosh, Paul Denny, Juho Leinonen, James Prather, Adish Singla

AI总结:

研究如何调整以提示为中心的编程活动,通过向正确解决方案注入真实可运行的错误,分析学生对自然发生的提示相关故障和故意注入错误的行为,发现学生对不同错误源反应不同,二者共同支持有用的生成式人工智能工作流程。

摘要:

随着生成式人工智能(GenAI)在软件开发中变得越来越核心,计算机科学教育正在整合以提示为中心的工作流程,学生用自然语言描述预期程序行为以获取代码。然而,专业实践需要仔细审查和验证GenAI生成的代码,这对CS1级活动构成挑战。我们研究如何调整以提示为中心的编程活动,具体探索一种方法,将现实、可运行的错误注入到正确的解决方案中,要求学生阅读和修复生成的输出。我们分析了917名学生的2636个会话,并检查了每个会话中自然发生的提示相关故障和故意注入错误的实例的行为。我们的研究结果表明,学生对不同的错误源有不同的反应。故意注入的错误更常导致直接的代码编辑和更高的下一次尝试成功率,表明对差点成功的解决方案进行局部修复。与提示相关的故障反而更常导致学生通过澄清约束、更新函数签名、添加边缘情况或重新构建任务来改进提示。学生的反思强化了对审查和修复的重视,描述了在代码理解、代码审查和调试方面的有用实践,以及更谨慎的验证思维和对GenAI局限性的更高认识。最终,与提示相关的故障和注入的错误共同支持了一种对教学有用的GenAI工作流程,学生通过提示进行规范细化,并通过代码编辑进行调试。

英文摘要:

As Generative AI (GenAI) becomes increasingly central to software development, CS education is integrating prompt-centered workflows where students describe intended program behavior in natural language to elicit code. However, professional practice requires careful review and verification of GenAI-generated code that may appear correct while containing subtle faults. This creates a challenge for CS1-level activities, where current models often solve tasks correctly and reduce students' incentive to closely inspect generated outputs. We investigate how prompt-centered programming activities can be adapted to better foster these practices. Specifically, we explore an approach where realistic, runnable bugs are injected into otherwise correct solutions, thus requiring students to read and repair generated outputs. We analyzed 2,636 sessions from 917 students, and examined behavior across instances of naturally occurring prompt-related failures and deliberately injected bugs within each session. Our findings show that students responded differently across bug sources. Deliberately injected bugs more often led to direct code edits and higher next-attempt success, suggesting localized repair of near-miss solutions. Prompt-related failures instead more often led students to refine prompts by clarifying constraints, updating function signatures, adding edge cases, or reframing the task. Student reflections reinforce the emphasis on review and repair, describing useful practice in code understanding, code review, and debugging, as well as a more careful verification mindset and greater awareness of GenAI limitations. Ultimately, prompt-related failures and injected bugs together support a pedagogically useful GenAI workflow, where students practice both specification refinement through prompts and debugging through code editing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑