FacialFlowNet:通过多样化数据集和解耦模型推进面部光流估计
FacialFlowNet: Advancing Facial Optical Flow Estimation with a Diverse Dataset and a Decomposed Model
AI总结:
该研究构建含9,635身份、105,970图像对的大规模面部光流数据集FFN,并提出可将光流解耦为头部与表情成分的DecFlow,显著提升光流估计及微表情识别性能。
摘要:
面部动作在传达态度和意图方面发挥着关键作用,而面部光流为其提供了动态且细粒度的表示。然而,数据集的稀缺以及现代基线的缺乏阻碍了面部光流研究的进展。本文提出了FacialFlowNet(FFN)这一新颖的大规模面部光流数据集,以及首个能够分解面部光流的方法——Decomposed Facial Flow Model(DecFlow)。FFN包含9,635个身份和105,970对图像,为细粒度面部与头部运动分析提供了前所未有的多样性。DecFlow具有面部语义感知编码器和解耦光流解码器,擅长准确估计面部光流并将其分解为头部成分和表情成分。大量实验表明,FFN显著提升了多种光流方法的面部光流估计准确率,端点误差(EPE)最高降低11%(从3.91降至3.48)。此外,DecFlow与FFN结合后,在合成场景和真实场景中均优于现有方法,并增强了面部表情分析。解耦得到的表情光流使微表情识别准确率大幅提升18%(从69.1%提升至82.1%)。这些贡献代表了面部运动分析和光流估计领域的重大进展。代码和数据集均可获取。
英文摘要:
Facial movements play a crucial role in conveying altitude and intentions, and facial optical flow provides a dynamic and detailed representation of it. However, the scarcity of datasets and a modern baseline hinders the progress in facial optical flow research. This paper proposes FacialFlowNet (FFN), a novel large-scale facial optical flow dataset, and the Decomposed Facial Flow Model (DecFlow), the first method capable of decomposing facial flow. FFN comprises 9,635 identities and 105,970 image pairs, offering unprecedented diversity for detailed facial and head motion analysis. DecFlow features a facial semantic-aware encoder and a decomposed flow decoder, excelling in accurately estimating and decomposing facial flow into head and expression components. Comprehensive experiments demonstrate that FFN significantly enhances the accuracy of facial flow estimation across various optical flow methods, achieving up to an 11% reduction in Endpoint Error (EPE) (from 3.91 to 3.48). Moreover, DecFlow, when coupled with FFN, outperforms existing methods in both synthetic and real-world scenarios, enhancing facial expression analysis. The decomposed expression flow achieves a substantial accuracy improvement of 18% (from 69.1% to 82.1%) in micro-expressions recognition. These contributions represent a significant advancement in facial motion analysis and optical flow estimation. Codes and datasets can be found.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。