arXiv 2303.078402023-03-14cs.CV

基于参考热图Transformer的精确面部关键点检测

Precise Facial Landmark Detection by Reference Heatmap Transformer

Zhongnan University of Economics and Law（中南财经政法大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人研究院）；China Three Gorges University（三峡大学）

Jun Wan, Jun Liu, Jie Zhou, Zhihui Lai, Linlin Shen, Hang Sun, Ping Xiong, Wenwen Min

备注：: Accepted by IEEE Transactions on Image Processing, March 2023
DOI：: 10.1109/TIP.2023.3261749

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Reference Heatmap Transformer（RHT），通过软硬变换模块和Multi-Scale Feature Fusion Module融合参考热图信息与语义特征，在复杂姿态、遮挡和光照条件下提升面部关键点检测精度，并在多个基准数据集上超越现有方法。

摘要：

大多数面部关键点检测方法通过将输入的面部外观特征映射到关键点热图来预测关键点，并已取得有前景的结果。然而，当人脸图像受到大姿态、严重遮挡和复杂光照影响时，这些方法无法学习具有判别力的特征表示和有效的面部形状约束，也无法准确预测关键点热图中每个元素的值，从而限制了其检测精度。为了解决这一问题，我们提出了一种新颖的Reference Heatmap Transformer（RHT），通过引入参考热图信息来实现更精确的面部关键点检测。所提出的RHT由一个Soft Transformation Module（STM）和一个Hard Transformation Module（HTM）组成，二者可以相互协作，以促进参考热图信息和面部形状约束的准确变换。随后，我们提出了一个Multi-Scale Feature Fusion Module（MSFFM），用于融合变换后的热图特征与从原始人脸图像中学习到的语义特征，以增强特征表示，从而生成更准确的目标热图。据我们所知，这是首个探索如何通过变换参考热图信息来增强面部关键点检测的研究。在具有挑战性的基准数据集上的实验结果表明，我们提出的方法优于文献中的state-of-the-art方法。

英文摘要：

Most facial landmark detection methods predict landmarks by mapping the input facial appearance features to landmark heatmaps and have achieved promising results. However, when the face image is suffering from large poses, heavy occlusions and complicated illuminations, they cannot learn discriminative feature representations and effective facial shape constraints, nor can they accurately predict the value of each element in the landmark heatmap, limiting their detection accuracy. To address this problem, we propose a novel Reference Heatmap Transformer (RHT) by introducing reference heatmap information for more precise facial landmark detection. The proposed RHT consists of a Soft Transformation Module (STM) and a Hard Transformation Module (HTM), which can cooperate with each other to encourage the accurate transformation of the reference heatmap information and facial shape constraints. Then, a Multi-Scale Feature Fusion Module (MSFFM) is proposed to fuse the transformed heatmap features and the semantic features learned from the original face images to enhance feature representations for producing more accurate target heatmaps. To the best of our knowledge, this is the first study to explore how to enhance facial landmark detection by transforming the reference heatmap information. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。