基于属性引导的多模态深度伪造检测：跨模态取证指纹
Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints
AI总结：
本文提出AMDD框架，通过跨模态取证指纹一致性损失，引导模型学习生成器特定的取证特征，提升多模态深度伪造检测的准确性和鲁棒性。
摘要：
音频-视觉深度伪造已达到现实水平，使感知检测不可靠，威胁媒体完整性和生物识别安全。尽管多模态检测显示出潜力，但大多数方法是二元分类任务，通常依赖数据集特定的伪影而非真实的生成痕迹。我们认为，无法识别视频如何伪造的检测器很可能在学习错误信号。不同于二元检测，属性引导学习在共享嵌入空间施加更强的几何约束，迫使模型编码生成器特定的取证内容而非捷径。我们提出属性引导多模态深度伪造检测（AMDD）框架，联合学习检测和属性归因。AMDD将生成器归因视为结构化正则化，约束表示几何向取证有意义的特征发展。我们引入跨模态取证指纹一致性（CMFFC）损失，强制对齐视觉和音频流中的生成器诱导伪影。这利用了相干操纵在模态间留下相关痕迹的事实，基于语音和面部动作之间的物理耦合，合成流程通常会破坏。在架构上，我们配对ResNet50与时间注意力进行视觉编码，与预训练的ResNet18处理梅尔频谱图，弥补了先前模型中编码器容量的差距。在FakeAVCeleb上，AMDD实现了99.7%的平衡准确率和99.8%的AUC，以及95.9%的归因准确率。跨数据集评估在DeepfakeTIMIT、DFDM和LAV-DF上证实，真实视频检测具有良好的鲁棒性，而对未见过的生成器的伪造检测仍是一个开放挑战，我们深入分析了这一点。
英文摘要：
Audio-visual deepfakes have reached a level of realism that makes perceptual detection unreliable, threatening media integrity and biometric security. While multimodal detection has shown promise, most approaches are binary classification tasks that often latch onto dataset-specific artifacts rather than genuine generative traces. We argue that a detector incapable of identifying how a video was forged is likely learning the wrong signal. Unlike binary detection, attribution-guided learning imposes a stronger geometric constraint on the shared embedding space, forcing the model to encode generator-specific forensic content rather than shortcuts. We propose the Attribution-Guided Multimodal Deepfake Detection (AMDD) framework, which jointly learns to detect and attribute manipulation. AMDD treats generator attribution as a structured regularization that constrains representation geometry toward forensically meaningful features. We introduce a Cross-Modal Forensic Fingerprint Consistency (CMFFC) loss to enforce alignment between generator-induced artifacts in visual and audio streams. This exploits the fact that coherent manipulation leaves correlated traces across modalities, grounded in the physical coupling between speech and facial articulation that synthetic pipelines routinely disrupt. Architecturally, we pair a ResNet50 with temporal attention for visual encoding against a pretrained ResNet18 for mel spectrograms, closing the encoder capacity gap found in prior models. On FakeAVCeleb, AMDD achieves 99.7% balanced accuracy and 99.8% AUC with 95.9% attribution accuracy. Cross-dataset evaluation on DeepfakeTIMIT, DFDM, and LAV-DF confirms that real video detection generalizes robustly, while fake detection on unseen generators remains an open challenge that we analyze in depth.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
