arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.01256cs.CVcs.NE
收藏

基于光流的人脸运动自监督方法

Self-Supervised Approach for Facial Movement Based Optical Flow

American University of Sharjah(沙迦美国大学);Monash University(莫纳什大学);Indian Institute of Technology Ropar(印度理工学院罗巴尔校区)

Muhannad Alkaddour, Usman Tariq, Abhinav Dhall

摘要:

计算光流是计算机视觉中的一个基本问题。然而,基于深度学习的光流技术对于诸如在面部中发现的那些非刚性运动表现不佳,这主要是由于缺乏表示精细面部运动的训练数据。我们假设,学习光流的面部运动数据将提高质量的预测流的脸。这项工作的目的有三个方面:(1)探索自我监督技术,以生成人脸图像的光流地面真相;(2)计算基线结果的影响,使用人脸数据训练卷积神经网络(CNN)预测光流;和(3)使用学习的光流在微表情识别,以证明其有效性。我们使用BP 4D-Spontaneous数据集中的面部关键点生成光流地面实况。生成的光流用于训练FlowNetS架构,以在生成的数据集上测试其性能。在人脸图像上训练的FlowNetS的性能超过了其他光流CNN架构,证明了它的有用性。将我们的光流特征与其他基于STSTNet微表情分类器的方法进行了比较,结果表明,利用本文方法得到的光流特征在人脸表情分析中具有良好的应用前景。

英文摘要:

Computing optical flow is a fundamental problem in computer vision. However, deep learning-based optical flow techniques do not perform well for non-rigid movements such as those found in faces, primarily due to lack of the training data representing the fine facial motion. We hypothesize that learning optical flow on face motion data will improve the quality of predicted flow on faces. The aim of this work is threefold: (1) exploring self-supervised techniques to generate optical flow ground truth for face images; (2) computing baseline results on the effects of using face data to train Convolutional Neural Networks (CNN) for predicting optical flow; and (3) using the learned optical flow in micro-expression recognition to demonstrate its effectiveness. We generate optical flow ground truth using facial key-points in the BP4D-Spontaneous dataset. The generated optical flow is used to train the FlowNetS architecture to test its performance on the generated dataset. The performance of FlowNetS trained on face images surpassed that of other optical flow CNN architectures, demonstrating its usefulness. Our optical flow features are further compared with other methods using the STSTNet micro-expression classifier, and the results indicate that the optical flow obtained using this work has promising applications in facial expression analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑