解读儿童绘本中的情绪：多模态大语言模型在教育应用中的对比分析
Deciphering Emotions in Children Storybooks: A Comparative Analysis of Multimodal LLMs in Educational Applications
AI总结：
本研究针对阿拉伯语语境下儿童绘本情绪识别研究不足的问题，对比了GPT-4o与Gemini 1.5 Pro在三种提示策略下的表现，发现GPT-4o更优但二者均存在文化理解局限，为开发适配阿拉伯文化的情绪感知教育技术提供参考。
摘要：
多模态AI系统的情绪识别能力对于开发具有文化适应性的教育技术至关重要，但在阿拉伯语语境下相关研究仍显不足，而该地区正迫切需要契合文化背景的学习工具。本研究评估了两款先进多模态大语言模型——GPT-4o和Gemini 1.5 Pro——在处理阿拉伯语儿童绘本插图时的情绪识别性能。我们采用来自7本阿拉伯语绘本的75张图像，在三种提示策略（零样本、少样本和思维链）下对两款模型进行评估，并基于Plutchik情绪框架将模型预测结果与人工标注进行对比。GPT-4o在所有条件下的表现均 consistently 优于Gemini，在思维链提示下取得了59%的最高宏F1值，而Gemini的最佳表现仅为43%。错误分析揭示了系统性的误分类模式，其中效价反转占错误总数的60.7%，同时两款模型都难以处理具有文化细微差别的情绪和模糊的叙事语境。这些发现凸显了当前模型在文化理解方面的根本局限性，也强调了需采用具备文化敏感性的训练方法，为阿拉伯语学习者开发有效的情绪感知教育技术。
英文摘要：
Emotion recognition capabilities in multimodal AI systems are crucial for developing culturally responsive educational technologies, yet remain underexplored for Arabic language contexts where culturally appropriate learning tools are critically needed. This study evaluates the emotion recognition performance of two advanced multimodal large language models, GPT-4o and Gemini 1.5 Pro, when processing Arabic children's storybook illustrations. We assessed both models across three prompting strategies (zero-shot, few-shot, and chain-of-thought) using 75 images from seven Arabic storybooks, comparing model predictions with human annotations based on Plutchik's emotional framework. GPT-4o consistently outperformed Gemini across all conditions, achieving the highest macro F1-score of 59% with chain-of-thought prompting compared to Gemini's best performance of 43%. Error analysis revealed systematic misclassification patterns, with valence inversions accounting for 60.7% of errors, while both models struggled with culturally nuanced emotions and ambiguous narrative contexts. These findings highlight fundamental limitations in current models' cultural understanding and emphasize the need for culturally sensitive training approaches to develop effective emotion-aware educational technologies for Arabic-speaking learners.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
