评估视觉语言模型在情感识别中的表现
Evaluating Vision-Language Models for Emotion Recognition
AI总结：
针对VLMs情感处理能力缺乏详细评估的问题，本研究首次全面评估其从图像识别诱发情感的表现，创建相关基准并分析正确性与鲁棒性，为后续研究提供指导。
摘要：
大型视觉语言模型（VLMs）在多项客观多模态推理任务中取得了前所未有的成功。然而，为进一步增强其与人类共情且有效的沟通能力，提升VLMs处理和理解情感的方式至关重要。尽管情感理解的改进受到大量研究关注，但针对情感相关任务的VLMs详细评估仍存在不足，而此类评估可能为下游微调工作提供指导。本研究首次对VLMs从图像中识别诱发情感的能力进行全面评估，创建了诱发情感识别任务的基准，并从正确性和鲁棒性角度研究VLMs在该任务中的表现。通过多项实验，我们验证了情感识别表现所依赖的关键因素，同时分析了VLMs在过程中产生的各类错误。最后，我们通过人类评估研究指出错误的潜在原因，并基于实验结果为VLMs情感研究的未来方向提供建议。
英文摘要：
Large Vision-Language Models (VLMs) have achieved unprecedented success in several objective multimodal reasoning tasks. However, to further enhance their capabilities of empathetic and effective communication with humans, improving how VLMs process and understand emotions is crucial. Despite significant research attention on improving affective understanding, there is a lack of detailed evaluations of VLMs for emotion-related tasks, which can potentially help inform downstream fine-tuning efforts. In this work, we present the first comprehensive evaluation of VLMs for recognizing evoked emotions from images. We create a benchmark for the task of evoked emotion recognition and study the performance of VLMs for this task, from perspectives of correctness and robustness. Through several experiments, we demonstrate important factors that emotion recognition performance depends on, and also characterize the various errors made by VLMs in the process. Finally, we pinpoint potential causes for errors through a human evaluation study. We use our experimental results to inform recommendations for the future of emotion research in the context of VLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
