CUPID: 重构UV纹理图用于可解释的特定人物深度伪造检测
CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection
AI总结：
提出CUPID方法，利用3D人脸重建的UV纹理图和掩码自编码器，无需深度伪造视频训练即可检测特定人物深度伪造，并实现可解释性和鲁棒性。
摘要：
针对高知名度人物（Person-of-Interest, POI）的深度伪造对现代民主社会构成威胁。当前的POI深度伪造检测方法在鲁棒性、效率和可解释性方面仍存在不足。本文提出CUPID，一种POI视频深度伪造检测器，结合了UV纹理图（源自3D人脸重建的面部外观表示）和掩码自编码器（MAE）的表征学习能力。我们的方法在训练阶段不需要任何深度伪造视频，甚至无需在训练集中包含特定POI：从真实视频帧中提取的UV纹理图与MAE上下文引导重构相结合，产生的潜在空间能够捕获丰富且具有判别性的面部特征，即使对于训练中未见过的身份也是如此。在测试阶段，从描述POI的查询视频中提取的嵌入可以与原始参考视频进行匹配，以评估视频真实性。此外，在UV空间中操作自然提供了额外的可解释性层。具体来说，我们可以提取解码残差图，突出显示测试视频中哪些面部区域与相应POI的身份表示偏差最大。在四个深度伪造数据集上的实验表明，CUPID在大多数数据集上优于当前最先进方法，并在强下采样和压缩下实现了最佳的整体鲁棒性，同时提供了更快的推理速度。我们的实验代码将在以下网址发布：https://this https URL。
英文摘要：
Deepfakes targeting a high-profile individual, known as Person-of-Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post-processing, efficiency and interpretability, key aspects of modern deepfake detectors. In this paper we propose CUPID, a POI video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake videos in its training phase. Moreover, it does not even require including a specific POI in the training set: the combination of UV texture maps extracted from real video frames and the MAE context-guided reconstruction yields a latent space that captures rich and discriminative facial features even for identities unseen during training. In the testing phase, the embeddings extracted from a query video depicting the POI can be matched against pristine reference videos to assess the video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake datasets show that CUPID outperforms the current state of the art on most datasets and achieves the best overall robustness against strong downscaling and compression, while also providing substantially faster inference. Our experimental code will be released at https://github.com/polimi-ispl/CUPID.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
