面部标志点的实时形状跟踪
Real-Time Shape Tracking of Facial Landmarks
AI总结：
本文提出结合SegNet语义分割与KLT点跟踪器的方法，通过定期执行SegNet解决点丢失问题并降低计算开销，实现面部标志点精确形状的实时跟踪。
摘要：
面部标志点的检测及其形状的准确跟踪在实时虚拟化妆应用中至关重要，用户可以通过朝不同方向移动面部来查看化妆效果。典型的面部跟踪技术会检测多个面部标志点，并使用诸如Kanade-Lucas-Tomasi（KLT）点跟踪器之类的点跟踪器对其进行跟踪。通常，跟踪面部使用5个或64个点。尽管这些点足以跟踪面部标志点的大致位置，但不足以跟踪面部标志点的精确形状。在本文中，我们提出了一种结合深度学习技术和点跟踪器的方法，能够实时跟踪面部标志点的精确形状。我们使用SegNet准确检测面部标志点，SegNet基于深度学习执行语义分割。检测到的标志点的边缘点使用KLT点跟踪器进行跟踪。尽管KLT点跟踪器很流行，但它存在点丢失问题。我们通过定期执行SegNet来计算面部标志点的形状，从而解决这一问题。也就是说，通过结合这两种技术，我们可以避免SegNet在实时形状跟踪中的计算开销，并避免KLT点跟踪器的点丢失问题。我们进行了若干实验来评估所提方法的性能，并在本文中报告了部分结果。
英文摘要：
Detection of facial landmarks and accurate tracking of their shape are essential in real-time virtual makeup applications, where users can see the makeups effect by moving their face in different directions. Typical face tracking techniques detect diverse facial landmarks and track them using a point tracker such as the Kanade-Lucas-Tomasi (KLT) point tracker. Typically, 5 or 64 points are used for tracking a face. Even though these points are sufficient to track the approximate locations of facial landmarks, they are not sufficient to track the exact shape of facial landmarks. In this paper, we propose a method that can track the exact shape of facial landmarks in real-time by combining a deep learning technique and a point tracker. We detect facial landmarks accurately using SegNet, which performs semantic segmentation based on deep learning. Edge points of detected landmarks are tracked using the KLT point tracker. In spite of its popularity, the KLT point tracker suffers from the point loss problem. We solve this problem by executing SegNet periodically to calculate the shape of facial landmarks. That is, by combining the two techniques, we can avoid the computational overhead of SegNet for real-time shape tracking and the point loss problem of the KLT point tracker. We performed several experiments to evaluate the performance of our method and report some of the results herein.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
