多模态人工智能系统在多学科物理概念量表上的多语言表现
Multilingual Performance of a Multimodal Artificial Intelligence System on Multisubject Physics Concept Inventories
AI总结:
本研究评估了 GPT-4o 在多语言、多学科物理概念量表上的多模态表现,发现其除实验室技能外优于本科生,但视觉解读能力较弱,并强调了物理教育中应对 AI 的策略。
摘要:
我们使用涵盖多种语言和学科类别的多样化物理概念量表,调查了基于大型语言模型的人工智能(AI)系统 GPT-4o 的多语言和多模态表现。这些量表源自 PhysPort 网站,涵盖经典物理主题(如力学、电磁学、光学和热力学),以及相对论、量子力学、天文学、数学和实验室技能。与以往仅基于文本的研究不同,我们将量表作为图像上传,以反映学生在试卷上看到的内容,从而评估系统的多模态功能。结果表明,不同学科的表现存在差异,其中实验室技能表现最弱。我们还观察到不同语言间的差异,英语和欧洲语言表现最佳。值得注意的是,量表题目的相对难度在很大程度上独立于调查所用的语言。将 AI 结果与关于学生表现的现有文献进行比较,我们发现,除实验室技能外,该 AI 系统在所有学科类别中的表现均优于授课后的本科生平均水平。此外,AI 在需要视觉图像解读的题目上表现不如纯文本题目。尽管我们的探索性发现表明 GPT-4o 在物理教育中具有潜在应用价值,但也强调了教师亟需培养学生批判性评估 AI 输出的能力,深思熟虑地调整课程以应对 AI 进步,并解决与 AI 整合相关的公平性问题。
英文摘要:
We investigate the multilingual and multimodal performance of a large language model-based artificial intelligence (AI) system, GPT-4o, using a diverse set of physics concept inventories spanning multiple languages and subject categories. The inventories, sourced from the PhysPort website, cover classical physics topics such as mechanics, electromagnetism, optics, and thermodynamics, as well as relativity, quantum mechanics, astronomy, mathematics, and laboratory skills. Unlike previous text-only studies, we uploaded the inventories as images to reflect what a student would see on paper, thereby assessing the system's multimodal functionality. Our results indicate variation in performance across subjects, with laboratory skills standing out as the weakest. We also observe differences across languages, with English and European languages showing the strongest performance. Notably, the relative difficulty of an inventory item is largely independent of the language of the survey. When comparing AI results to existing literature on student performance, we find that the AI system outperforms average post-instruction undergraduate students in all subject categories except laboratory skills. Furthermore, the AI performs worse on items requiring visual interpretation of images than on those that are purely text-based. While our exploratory findings show GPT-4o's potential usefulness in physics education, they highlight the critical need for instructors to foster students' ability to critically evaluate AI outputs, adapt curricula thoughtfully in response to AI advancements, and address equity concerns associated with AI integration.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。