未反思的接受——探究ChatGPT辅助物理问题求解的负面后果
Unreflected Acceptance -- Investigating the Negative Consequences of ChatGPT-Assisted Problem Solving in Physics Education
AI总结：
本研究通过对比物理专业学生使用搜索引擎与ChatGPT解题的表现，发现学生过度信任ChatGPT导致近半错误答案被接受，且交互行为缺乏反思，凸显了指导LLM使用和提升用户认知的必要性。
摘要：
大型语言模型（LLMs）近年来广受欢迎。然而，它们通过ChatGPT的普遍可用性对教育等日常生活敏感领域的影响仍不明确。尽管如此，学生和教育工作者已经感受到了对既有教育方法的社会影响。我们的工作聚焦于高等物理教育，并考察问题求解策略。在一项研究中，具有物理背景的学生被分配去解决物理习题，其中一组可以使用互联网搜索引擎（N=12），另一组被允许使用ChatGPT（N=27）。我们评估了他们的表现、策略以及与所提供工具的交互。结果表明，在ChatGPT支持下提供的解决方案中，近一半被学生错误地认为是正确的，这表明即使在他们的专业领域内，他们也过度信任ChatGPT。同样，在42%的情况下，学生使用复制粘贴来查询ChatGPT——这种方法在搜索引擎查询中仅占4%——这凸显了两组之间交互行为的显著差异，并表明在使用ChatGPT时反思有限。在我们的工作中，我们证明了需要（1）指导学生如何与LLMs交互，以及（2）提高用户对潜在缺陷的认识。
英文摘要：
Large language models (LLMs) have recently gained popularity. However, the impact of their general availability through ChatGPT on sensitive areas of everyday life, such as education, remains unclear. Nevertheless, the societal impact on established educational methods is already being experienced by both students and educators. Our work focuses on higher physics education and examines problem solving strategies. In a study, students with a background in physics were assigned to solve physics exercises, with one group having access to an internet search engine (N=12) and the other group being allowed to use ChatGPT (N=27). We evaluated their performance, strategies, and interaction with the provided tools. Our results showed that nearly half of the solutions provided with the support of ChatGPT were mistakenly assumed to be correct by the students, indicating that they overly trusted ChatGPT even in their field of expertise. Likewise, in 42% of cases, students used copy & paste to query ChatGPT -- an approach only used in 4% of search engine queries -- highlighting the stark differences in interaction behavior between the groups and indicating limited reflection when using ChatGPT. In our work, we demonstrated a need to (1) guide students on how to interact with LLMs and (2) create awareness of potential shortcomings for users.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
