arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.05523cs.HCcs.CV
收藏

DeepFace-Attention:用于注意力估计的多模态人脸生物特征识别技术及其在电子学习中的应用

DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning

Universidad Autonoma de Madrid(马德里自治大学)

Roberto Daza, Luis F. Gomez, Julian Fierrez, Aythami Morales, Ruben Tolosana, Javier Ortega-Garcia

AI总结:

本研究提出DeepFace-Attention多模态注意力估计方法,融合多种人脸相关特征,在mEBAL2数据集上准确率超越现有SOTA,可应用于电子学习场景。

摘要:

本研究提出了一种创新的注意力水平(认知负荷)估计方法,该方法采用集成的人脸分析技术处理网络摄像头视频。我们的方法在电子学习等应用场景中尤为实用,因此我们在mEBAL2数据库(一个在电子学习环境中采集的公开多模态数据库)上对该方法进行了训练、评估与对比。mEBAL2数据库包含60名用户的数据,这些用户完成了8项不同难度的任务,任务难度的变化会导致用户认知负荷的改变。我们的方法适配了当前最先进的人脸分析技术,将用户的认知负荷量化为高注意力或低注意力两种等级。方法使用了多种与认知负荷相关的行为信号和生理过程特征,包括眨眼、心率、面部动作单元、头部姿态等。此外,我们开展了多项研究,探究哪些单一特征能取得更优效果、最高效的特征组合方式、局部与全局特征的作用,以及时间间隔对注意力水平估计的影响等问题。研究发现,全局人脸特征更适用于采用分数级融合的多模态系统,尤其是随着时间窗口增大时效果更明显;而局部特征更适合结合分数级融合方法,通过神经网络训练实现融合。在公开基准数据集mEBAL2上,我们的方法准确率超过了现有最先进水平。

英文摘要:

This work introduces an innovative method for estimating attention levels (cognitive load) using an ensemble of facial analysis techniques applied to webcam videos. Our method is particularly useful, among others, in e-learning applications, so we trained, evaluated, and compared our approach on the mEBAL2 database, a public multi-modal database acquired in an e-learning environment. mEBAL2 comprises data from 60 users who performed 8 different tasks. These tasks varied in difficulty, leading to changes in their cognitive loads. Our approach adapts state-of-the-art facial analysis technologies to quantify the users' cognitive load in the form of high or low attention. Several behavioral signals and physiological processes related to the cognitive load are used, such as eyeblink, heart rate, facial action units, and head pose, among others. Furthermore, we conduct a study to understand which individual features obtain better results, the most efficient combinations, explore local and global features, and how temporary time intervals affect attention level estimation, among other aspects. We find that global facial features are more appropriate for multimodal systems using score-level fusion, particularly as the temporal window increases. On the other hand, local features are more suitable for fusion through neural network training with score-level fusion approaches. Our method outperforms existing state-of-the-art accuracies using the public mEBAL2 benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑