无限3D Landmarks：改进连续2D人脸Landmark检测
Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detection
AI总结：
本文针对2D人脸landmark检测器的三大实际问题，提出联合训练空间变换网络实现最优人脸归一化、在规范3D空间推断landmark并预测相机参数以提升精度和获取3D形状，以及引入语义校正网络解决多数据集标注不一致问题，显著提升了检测准确性和时间稳定性。
摘要：
本文研究了最先进的人脸landmark检测器在实际应用中的3个重要问题，并展示了特定架构修改的组合如何直接提升其准确性和时间稳定性。首先，许多人脸landmark检测器需要将人脸归一化作为预处理步骤，这由一个单独训练的神经网络完成，该网络对输入图像中的人脸进行裁剪和缩放。无法保证这个预训练网络为人脸landmark检测执行最优的人脸归一化。我们转而分析使用spatial transformer network，以无监督方式与landmark检测器一起训练，并联合学习最优人脸归一化和landmark检测。其次，我们展示了修改landmark预测器的输出头以在规范3D空间中推断landmark可以进一步提高准确性。为了将预测的3D landmark转换为屏幕空间，我们额外从输入图像预测相机内参和头部姿态。作为一个附带好处，这允许仅使用2D landmark作为监督来从给定图像预测3D人脸形状，这在确定landmark可见性等方面很有用。最后，当同时在多个数据集上训练landmark检测器时，数据集间的标注不一致迫使网络产生次优的平均值。我们提出添加semantic correction network来解决这个问题。这个额外的轻量级神经网络与landmark检测器一起训练，不需要任何额外监督。虽然本文的见解可以应用于大多数常见的landmark检测器，但我们特别针对一个最近提出的连续2D landmark检测器，以展示我们的每项添加如何在标准基准上带来相对于最先进方法的有意义的改进。
英文摘要：
In this paper, we examine 3 important issues in the practical use of state-of-the-art facial landmark detectors and show how a combination of specific architectural modifications can directly improve their accuracy and temporal stability. First, many facial landmark detectors require face normalization as a preprocessing step, which is accomplished by a separately-trained neural network that crops and resizes the face in the input image. There is no guarantee that this pre-trained network performs the optimal face normalization for landmark detection. We instead analyze the use of a spatial transformer network that is trained alongside the landmark detector in an unsupervised manner, and jointly learn optimal face normalization and landmark detection. Second, we show that modifying the output head of the landmark predictor to infer landmarks in a canonical 3D space can further improve accuracy. To convert the predicted 3D landmarks into screen-space, we additionally predict the camera intrinsics and head pose from the input image. As a side benefit, this allows to predict the 3D face shape from a given image only using 2D landmarks as supervision, which is useful in determining landmark visibility among other things. Finally, when training a landmark detector on multiple datasets at the same time, annotation inconsistencies across datasets forces the network to produce a suboptimal average. We propose to add a semantic correction network to address this issue. This additional lightweight neural network is trained alongside the landmark detector, without requiring any additional supervision. While the insights of this paper can be applied to most common landmark detectors, we specifically target a recently-proposed continuous 2D landmark detector to demonstrate how each of our additions leads to meaningful improvements over the state-of-the-art on standard benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
