使用动作单元引导的视频表示检测局部深度伪造操纵
Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations
AI总结：
针对深度伪造视频中局部编辑难以检测的问题，提出基于面部动作单元引导时空表示的检测方法，融合pretext任务学习的表示，在多种场景下准确率超现有方法20%，鲁棒性强。
摘要：
随着生成建模的快速发展，深度伪造技术正日益缩小真实视频与合成视频之间的差距，引发了严重的隐私和安全问题。除了传统的换脸和重演，最近最先进的深度伪造生成方法中出现了一种新趋势，即局部编辑，例如对特定面部特征的细微操纵，如挑眉、改变眼睛形状或修改嘴巴表情。这些细粒度操纵对现有检测模型构成重大挑战，它们难以捕捉此类局部变化。据我们所知，这项工作提出了首个明确设计用于泛化检测深度伪造视频中局部编辑的方法，该方法利用面部动作单元引导的时空表示。我们的方法利用基于交叉注意力的融合，融合从随机掩蔽和动作单元检测等 pretext任务中学习到的表示，以创建有效编码细微局部变化的嵌入。在多种深度伪造生成方法上的综合评估表明，尽管我们的方法仅在传统FF+数据集上训练，但在检测具有细粒度局部编辑的最新深度伪造生成视频方面树立了新基准，准确率比当前最先进的检测方法提高了20%。此外，我们的方法在标准数据集上表现出竞争力，突显了其在各种局部和全局伪造类型中的鲁棒性和泛化能力。
英文摘要：
With rapid advancements in generative modeling, deepfake techniques are increasingly narrowing the gap between real and synthetic videos, raising serious privacy and security concerns. Beyond traditional face swapping and reenactment, an emerging trend in recent state-of-the-art deepfake generation methods involves localized edits such as subtle manipulations of specific facial features like raising eyebrows, altering eye shapes, or modifying mouth expressions. These fine-grained manipulations pose a significant challenge for existing detection models, which struggle to capture such localized variations. To the best of our knowledge, this work presents the first detection approach explicitly designed to generalize to localized edits in deepfake videos by leveraging spatiotemporal representations guided by facial action units. Our method leverages a cross-attention-based fusion of representations learned from pretext tasks like random masking and action unit detection, to create an embedding that effectively encodes subtle, localized changes. Comprehensive evaluations across multiple deepfake generation methods demonstrate that our approach, despite being trained solely on the traditional FF+ dataset, sets a new benchmark in detecting recent deepfake-generated videos with fine-grained local edits, achieving a $20\%$ improvement in accuracy over current state-of-the-art detection methods. Additionally, our method delivers competitive performance on standard datasets, highlighting its robustness and generalization across diverse types of local and global forgeries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
