评估开源视觉语言模型与传统深度学习模型在面部情绪识别中的表现
Evaluating Open-Source Vision Language Models for Facial Emotion Recognition against Traditional Deep Learning Models
AI总结：
本研究首次在 FER-2013 数据集上对比开源视觉语言模型与传统深度学习模型的面部情绪识别性能，提出结合 GFPGAN 图像恢复的评估流程，证明传统模型显著优于 VLM，并提供了计算成本分析与可复现基准。
摘要：
面部情绪识别（FER）对于人机交互和心理健康诊断等应用至关重要。本研究首次对开源视觉语言模型（VLM）——包括 Phi-3.5 Vision 和 CLIP——与传统深度学习模型 VGG19、ResNet-50 和 EfficientNet-B0 在具有挑战性的 FER-2013 数据集上进行了实证比较。该数据集包含 35,887 张低分辨率灰度图像，涵盖七种情绪类别。为解决 VLM 训练假设与 FER 数据噪声特性之间的不匹配问题，我们提出了一种新颖的流程，将基于 GFPGAN 的图像恢复与 FER 评估相结合。结果表明，传统模型，尤其是 EfficientNet-B0（86.44%）和 ResNet-50（85.72%），显著优于 CLIP（64.07%）和 Phi-3.5 Vision（51.66%）等 VLM，凸显了 VLM 在低质量视觉任务中的局限性。除了使用精确率、召回率、F1 分数和准确率进行性能评估外，我们还提供了涵盖预处理、训练、推理和评估阶段的详细计算成本分析，为部署提供了实用见解。本研究强调了使 VLM 适应噪声环境的必要性，并为未来情绪识别研究提供了一个可复现的基准。
英文摘要：
Facial Emotion Recognition (FER) is crucial for applications such as human-computer interaction and mental health diagnostics. This study presents the first empirical comparison of open-source Vision-Language Models (VLMs), including Phi-3.5 Vision and CLIP, against traditional deep learning models VGG19, ResNet-50, and EfficientNet-B0 on the challenging FER-2013 dataset, which contains 35,887 low-resolution grayscale images across seven emotion classes. To address the mismatch between VLM training assumptions and the noisy nature of FER data, we introduce a novel pipeline that integrates GFPGAN-based image restoration with FER evaluation. Results show that traditional models, particularly EfficientNet-B0 (86.44%) and ResNet-50 (85.72%), significantly outperform VLMs like CLIP (64.07%) and Phi-3.5 Vision (51.66%), highlighting the limitations of VLMs in low-quality visual tasks. In addition to performance evaluation using precision, recall, F1-score, and accuracy, we provide a detailed computational cost analysis covering preprocessing, training, inference, and evaluation phases, offering practical insights for deployment. This work underscores the need for adapting VLMs to noisy environments and provides a reproducible benchmark for future research in emotion recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
