SVFAP：自监督视频面部情感感知器
SVFAP: Self-supervised Video Facial Affect Perceiver
AI总结：
本文提出自监督视频面部情感感知器SVFAP，通过掩码面部视频自编码与时间金字塔、空间瓶颈Transformer在大规模无标注视频上预训练，在九个数据集上显著超越以往方法。
摘要：
基于视频的面部情感分析因其在人机交互中的关键作用，近年来受到越来越多的关注。以往的研究主要集中于开发各种深度学习架构并以全监督方式进行训练。尽管这些监督方法已取得显著进展，但长期以来缺乏大规模高质量标注数据严重阻碍了其进一步发展。受近期自监督学习在计算机视觉领域成功的启发，本文提出了一种自监督方法，称为自监督视频面部情感感知器（SVFAP），以解决监督方法所面临的困境。具体而言，SVFAP利用掩码面部视频自编码在大规模无标注面部视频上进行自监督预训练。考虑到面部视频中存在大量时空冗余，我们提出了一种新颖的时间金字塔和空间瓶颈Transformer作为SVFAP的编码器，这不仅大幅降低了计算成本，而且取得了优异的性能。为验证我们方法的有效性，我们在涵盖三个下游任务的九个数据集上进行了实验，包括动态面部表情识别、维度情感识别和人格识别。综合结果表明，SVFAP能够通过大规模自监督预训练学习到强大的情感相关表征，并且在所有数据集上显著优于以往最先进的方法。代码可在https://github.com/sunlicai/SVFAP获取。
英文摘要：
Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing various deep learning architectures and training them in a fully supervised manner. Although significant progress has been achieved by these supervised methods, the longstanding lack of large-scale high-quality labeled data severely hinders their further improvements. Motivated by the recent success of self-supervised learning in computer vision, this paper introduces a self-supervised approach, termed Self-supervised Video Facial Affect Perceiver (SVFAP), to address the dilemma faced by supervised methods. Specifically, SVFAP leverages masked facial video autoencoding to perform self-supervised pre-training on massive unlabeled facial videos. Considering that large spatiotemporal redundancy exists in facial videos, we propose a novel temporal pyramid and spatial bottleneck Transformer as the encoder of SVFAP, which not only largely reduces computational costs but also achieves excellent performance. To verify the effectiveness of our method, we conduct experiments on nine datasets spanning three downstream tasks, including dynamic facial expression recognition, dimensional emotion recognition, and personality recognition. Comprehensive results demonstrate that SVFAP can learn powerful affect-related representations via large-scale self-supervised pre-training and it significantly outperforms previous state-of-the-art methods on all datasets. Code is available at https://github.com/sunlicai/SVFAP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
