arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1704.03067cs.CV
收藏

基于区域自适应、多标记学习和最优时间融合的动作单元检测

Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusing

CUNY City College(纽约市立大学城市学院);CUNY Graduate Center(纽约市立大学研究生中心)

Wei Li, Farnaz Abitahi, Zhigang Zhu

摘要:

动作单元(AU)检测对于面部分析至关重要。许多提出的方法面临着挑战性的问题,在处理不同的人脸区域的对齐,在时间信息的有效融合,并在训练多个AU标签的模型。为了更好地解决这些问题,我们提出了一个用于AU检测的深度学习框架,该框架具有感兴趣区域(ROI)自适应、集成多标签学习和基于最优LSTM的时间融合。首先,ROI裁剪网络(ROI Nets)旨在确保独立学习特定感兴趣的面部区域;每个子区域都有一个局部卷积神经网络(CNN)-ROI Net,其卷积滤波器仅针对相应区域进行训练。其次,采用多标签学习来整合这些单个ROI裁剪网络的输出,学习各个AU之间的相互关系,并获得用于AU检测的跨子区域的全局特征。最后,进行多个LSTM层的最佳选择以形成最佳LSTM Net,以最佳地融合时间特征,从而使AU预测最准确。在两个流行的AU检测数据集BP 4D和DISFA上对所提出的方法进行了评估,其性能显著优于现有技术,BP 4D和DISFA的平均改进分别约为13%和25%。

英文摘要:

Action Unit (AU) detection becomes essential for facial analysis. Many proposed approaches face challenging problems in dealing with the alignments of different face regions, in the effective fusion of temporal information, and in training a model for multiple AU labels. To better address these problems, we propose a deep learning framework for AU detection with region of interest (ROI) adaptation, integrated multi-label learning, and optimal LSTM-based temporal fusing. First, ROI cropping nets (ROI Nets) are designed to make sure specifically interested regions of faces are learned independently; each sub-region has a local convolutional neural network (CNN) - an ROI Net, whose convolutional filters will only be trained for the corresponding region. Second, multi-label learning is employed to integrate the outputs of those individual ROI cropping nets, which learns the inter-relationships of various AUs and acquires global features across sub-regions for AU detection. Finally, the optimal selection of multiple LSTM layers to form the best LSTM Net is carried out to best fuse temporal features, in order to make the AU prediction the most accurate. The proposed approach is evaluated on two popular AU detection datasets, BP4D and DISFA, outperforming the state of the art significantly, with an average improvement of around 13% on BP4D and 25% on DISFA, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑