arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1807.08775cs.HC
收藏

移动设备上基于CNN的人脸情感分析

CNN-based Facial Affect Analysis on Mobile Devices

Charlie Hewitt, Hatice Gunes

摘要:

本文重点介绍了用于移动设备面部情感分析的卷积神经网络(CNN)架构的设计、部署和评估。与传统的CNN方法不同,部署到移动设备的模型必须在保持高性能的同时最大限度地减少存储需求。因此,我们提出了三种已建立的CNN架构的变体,并在一个大型的、野外的面部图像基准数据集上对它们进行了比较评估。我们的研究结果表明,所提出的架构保留了类似的性能数据集基线,同时最大限度地减少存储要求:实现58%的准确度为8类情绪分类和平均RMSE为0.39的效价/唤醒预测。为了证明部署这些模型在现实世界中的应用的可行性,我们实现了一个音乐推荐界面的基础上预测的用户影响。虽然CNN模型没有在音乐推荐的背景下进行训练,但我们的案例研究表明:(i)经过训练的模型实现了与基准数据集相似的预测性能,以及(ii)用户倾向于积极评价界面提供的歌曲推荐。在iPhone 6S上部署的模型的平均运行时间相当于约45 fps,这表明所提出的架构也非常适合视频流的实时部署。

英文摘要:

This paper focuses on the design, deployment and evaluation of Convolutional Neural Network (CNN) architectures for facial affect analysis on mobile devices. Unlike traditional CNN approaches, models deployed to mobile devices must minimise storage requirements while retaining high performance. We therefore propose three variants of established CNN architectures and comparatively evaluate them on a large, in-the-wild benchmark dataset of facial images. Our results show that the proposed architectures retain similar performance to the dataset baseline while minimising storage requirements: achieving 58% accuracy for eight-class emotion classification and average RMSE of 0.39 for valence/arousal prediction. To demonstrate the feasibility of deploying these models for real-world applications, we implement a music recommendation interface based on predicted user affect. Although the CNN models were not trained in the context of music recommendation, our case study shows that: (i) the trained models achieve similar prediction performance to the benchmark dataset, and (ii) users tend to positively rate the song recommendations provided by the interface. Average runtime of the deployed models on an iPhone 6S equates to ~45 fps, suggesting that the proposed architectures are also well suited for real-time deployment on video streams.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑