面向面部行为理解的特征表示学习与身份对抗训练
Representation Learning and Identity Adversarial Training for Facial Behavior Understanding
AI总结：
本文提出Face9M大规模面部数据集与身份对抗训练方法FMAE-IAT，通过预训练掩码自编码器并抑制身份捷径学习，在多个AU检测基准上取得新的最先进性能。
摘要：
面部动作单元（AU）检测因能将复杂面部表情分解为单个肌肉运动而受到广泛关注。本文重新审视AU检测中的两个基本因素：多样且大规模的数据以及受试者身份正则化。受基础模型近期进展的启发，我们强调数据的重要性，并引入Face9M，这是一个包含来自多个公开来源的900万张面部图像的多样化数据集。在Face9M上预训练掩码自编码器可在AU检测和面部表情任务中获得强性能。更重要的是，我们强调身份对抗训练（IAT）在AU任务中尚未得到充分探索。为填补这一空白，我们首先表明AU数据集中的受试者身份会导致模型产生捷径学习，并导致AU预测的次优解。其次，我们证明学习身份不变特征需要强IAT正则化。最后，我们阐明了IAT的设计空间，并通过实验表明IAT能够规避基于身份的捷径学习，从而得到更好的解。我们提出的方法——面部掩码自编码器（FMAE）和IAT——简单、通用且有效。值得注意的是，所提出的FMAE-IAT方法在BP4D（67.1%）、BP4D+（66.8%）和DISFA（70.1%）数据库上取得了新的最先进F1分数，显著优于先前工作。我们在https://github.com/forever208/FMAE-IAT发布代码和模型。
英文摘要：
Facial Action Unit (AU) detection has gained significant attention as it enables the breakdown of complex facial expressions into individual muscle movements. In this paper, we revisit two fundamental factors in AU detection: diverse and large-scale data and subject identity regularization. Motivated by recent advances in foundation models, we highlight the importance of data and introduce Face9M, a diverse dataset comprising 9 million facial images from multiple public sources. Pretraining a masked autoencoder on Face9M yields strong performance in AU detection and facial expression tasks. More importantly, we emphasize that the Identity Adversarial Training (IAT) has not been well explored in AU tasks. To fill this gap, we first show that subject identity in AU datasets creates shortcut learning for the model and leads to sub-optimal solutions to AU predictions. Secondly, we demonstrate that strong IAT regularization is necessary to learn identity-invariant features. Finally, we elucidate the design space of IAT and empirically show that IAT circumvents the identity-based shortcut learning and results in a better solution. Our proposed methods, Facial Masked Autoencoder (FMAE) and IAT, are simple, generic and effective. Remarkably, the proposed FMAE-IAT approach achieves new state-of-the-art F1 scores on BP4D (67.1\%), BP4D+ (66.8\%), and DISFA (70.1\%) databases, significantly outperforming previous work. We release the code and model at https://github.com/forever208/FMAE-IAT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
