物理学生如何评价人工智能对理解性问题的回答？一项关于感知科学准确性与语言质量的研究
How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality
AI总结：
本研究通过102名物理学生对ChatGPT回答的评估，发现其语言质量与样本解答相近，但科学准确性评分较低，且差异随学生自评知识水平增加而增大，提示需谨慎评估AI回答并开展批判性思维训练。
摘要：
本研究旨在评估学生如何感知ChatGPT对物理理解性问题回答的语言质量和科学准确性。共有102名大一和大二物理学生被要求回答三个难度递增的入门力学问题（滚动运动、波动和流体动力学）。每个问题都配有四种不同的回答。所有回答均被标注为ChatGPT生成，但实际上其中一份样本解答由研究人员创建。本研究中获得的所有ChatGPT回答均存在错误、不精确、不完整或具有误导性。我们发现，ChatGPT回答与样本解答在感知语言质量上的差异很小。然而，学生对回答整体科学准确性的评分存在显著差异，对于低难度和中等难度的问题，样本解答被评为最佳。样本解答与ChatGPT回答之间的差异随着学生对问题内容自评知识水平的提高而增大。对于大多数学生不了解的最高难度问题（流体动力学），一个ChatGPT回答的评分与样本解答相当。因此，本研究提供了关于学生对ChatGPT回答的感知及其影响因素的实验数据。结果强调了教师和学生都需要仔细评估ChatGPT回答，特别是在科学准确性方面。因此，未来研究可以探索类似“识别机器人”活动在物理教育中的潜力，以培养学生的批判性思维能力。
英文摘要：
This study aimed at evaluating how students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics comprehension questions. A total of 102 first- and second-year physics students were confronted with three questions of progressing difficulty from introductory mechanics (rolling motion, waves, and fluid dynamics). Each question was presented with four different responses. All responses were attributed to ChatGPT, but in reality one sample solution was created by the researchers. All ChatGPT responses obtained in this study were wrong, imprecise, incomplete, or misleading. We found little differences in the perceived linguistic quality between ChatGPT responses and the sample solution. However, the students rated the overall scientific accuracy of the responses significantly differently, with the sample solution being rated best for the questions of low and medium difficulty. The discrepancy between the sample solution and the ChatGPT responses increased with the level of self-assessed knowledge of the question content. For the question of highest difficulty (fluid dynamics) that was unknown to most students, a ChatGPT response was rated just as good as the sample solution. Thus, this study provides data on the students' perception of ChatGPT responses and the factors influencing their perception. The results highlight the need for careful evaluation of ChatGPT responses both by instructors and students, particularly regarding scientific accuracy. Therefore, future research could explore the potential of similar "spot the bot"-activities in physics education to foster students' critical thinking skills.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
