Gemini Pro 在教育领域被 GPT-4V 击败：来自教育的证据
Gemini Pro Defeated by GPT-4V: Evidence from Education
AI总结：
本研究通过视觉问答和 NERIF 提示方法比较 Gemini Pro 与 GPT-4V 在教育场景中自动评分学生绘图的表现，发现 GPT-4V 在评分准确性和二次加权 Kappa 上显著优于 Gemini Pro，更适合多模态教育任务。
摘要：
本研究比较了 Gemini Pro 和 GPT-4V 在教育场景中的分类性能。研究采用视觉问答（VQA）技术，考察了两个模型读取基于文本的评分标准，并自动对科学教育中学生绘制的模型进行评分的能力。我们使用来自学生绘制的科学模型的数据集，并采用 NERIF（Notation-Enhanced Rubrics for Image Feedback，符号增强的图像反馈评分标准）提示方法，进行了定量和定性分析。研究结果表明，GPT-4V 在评分准确性和二次加权 Kappa 系数方面显著优于 Gemini Pro。定性分析显示，差异可能源于模型处理图像中细粒度文本的能力以及整体图像分类性能。即使通过进一步缩小输入图像尺寸来调整 NERIF 方法，Gemini Pro 似乎仍无法表现得与 GPT-4V 一样好。研究结果表明，GPT-4V 在处理复杂的多模态教育任务方面具有卓越的能力。研究结论认为，尽管两个模型都代表了人工智能的进步，但 GPT-4V 更高的性能使其成为涉及多模态数据解读的教育应用中更合适的工具。
英文摘要：
This study compared the classification performance of Gemini Pro and GPT-4V in educational settings. Employing visual question answering (VQA) techniques, the study examined both models' abilities to read text-based rubrics and then automatically score student-drawn models in science education. We employed both quantitative and qualitative analyses using a dataset derived from student-drawn scientific models and employing NERIF (Notation-Enhanced Rubrics for Image Feedback) prompting methods. The findings reveal that GPT-4V significantly outperforms Gemini Pro in terms of scoring accuracy and Quadratic Weighted Kappa. The qualitative analysis reveals that the differences may be due to the models' ability to process fine-grained texts in images and overall image classification performance. Even adapting the NERIF approach by further de-sizing the input images, Gemini Pro seems not able to perform as well as GPT-4V. The findings suggest GPT-4V's superior capability in handling complex multimodal educational tasks. The study concludes that while both models represent advancements in AI, GPT-4V's higher performance makes it a more suitable tool for educational applications involving multimodal data interpretation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
