arXiv 2407.156172024-07-22cs.CVcs.AI

Norface：通过身份归一化改进面部表情分析

Norface: Improving Facial Expression Analysis by Identity Normalization

Tongji University（同济大学）；Netease Fuxi AI Lab（网易伏羲人工智能实验室）；Happy Elements（乐元素科技）；Hebei Agricultural University（河北农业大学）；School of Computer Science, Hangzhou Dianzi University（杭州电子科技大学计算机学院）

Hanwei Liu, Rudong An, Zhimeng Zhang, Bowen Ma, Wei Zhang, Yan Song, Yujing Hu, Wei Chen, Yu Ding

备注：: Accepted by ECCV2024

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Norface框架，通过身份归一化去除面部表情分析中的任务无关噪声，结合专家混合网络，在AU检测、强度估计和情绪识别任务上超越现有最先进方法。

摘要：

面部表情分析由于存在意外的任务无关噪声（如身份、头部姿态和背景）而仍然具有挑战性。为解决这一问题，本文提出了一种新颖的框架，称为Norface，该框架统一用于动作单元（AU）分析和面部情绪识别（FER）任务。Norface由一个归一化网络和一个分类网络组成。首先，精心设计的归一化网络通过保持面部表情一致性，但将所有原始图像归一化为具有一致姿态和背景的共同身份，努力直接去除上述任务无关噪声。然后，这些额外的归一化图像被输入到分类网络。由于一致的身份和其他因素（如头部姿态、背景等），归一化图像使分类网络能够更有效地提取有用的表情信息。此外，分类网络结合了专家混合（Mixture of Experts）来细化潜在表示，包括处理面部表示的输入和多个（AU或情绪）标签的输出。大量实验验证了具有身份归一化见解的精心设计框架。所提出的方法在多个面部表情分析任务中优于现有最先进（SOTA）方法，包括AU检测、AU强度估计和FER任务，以及它们的跨数据集任务。对于归一化数据集和代码，请访问{https://norface-fea.github.io/}。

英文摘要：

Facial Expression Analysis remains a challenging task due to unexpected task-irrelevant noise, such as identity, head pose, and background. To address this issue, this paper proposes a novel framework, called Norface, that is unified for both Action Unit (AU) analysis and Facial Emotion Recognition (FER) tasks. Norface consists of a normalization network and a classification network. First, the carefully designed normalization network struggles to directly remove the above task-irrelevant noise, by maintaining facial expression consistency but normalizing all original images to a common identity with consistent pose, and background. Then, these additional normalized images are fed into the classification network. Due to consistent identity and other factors (e.g. head pose, background, etc.), the normalized images enable the classification network to extract useful expression information more effectively. Additionally, the classification network incorporates a Mixture of Experts to refine the latent representation, including handling the input of facial representations and the output of multiple (AU or emotion) labels. Extensive experiments validate the carefully designed framework with the insight of identity normalization. The proposed method outperforms existing SOTA methods in multiple facial expression analysis tasks, including AU detection, AU intensity estimation, and FER tasks, as well as their cross-dataset tasks. For the normalized datasets and code please visit {https://norface-fea.github.io/}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。