CAST：跨注意力时空特征融合用于深度伪造检测
CAST: Cross-Attentive Spatio-Temporal feature fusion for deepfake detection
AI总结：
本文提出CAST模型，通过跨注意力机制融合时空特征，提升深度伪造检测的鲁棒性与精度，实验表明在多个数据集上均取得优异性能。
摘要：
深度伪造已成为数字媒体真实性的重要威胁，亟需更先进的检测技术来识别微妙且时间依赖的篡改。CNNs擅长捕捉空间伪影，Transformers擅长建模时间不一致。然而，现有CNN-Transformer模型通常独立处理时空特征，尤其基于注意力的方法常使用独立机制并采用简单方式合并，限制了时空交互的深度。为解决此问题，我们提出CAST模型，利用跨注意力机制更有效地融合时空特征。我们的方法允许时间特征动态关注相关空间区域，增强模型检测细微、时间演变伪影（如闪烁眼睛或扭曲嘴唇）的能力。此设计使定位更精确，上下文理解更深入，从而在多样且挑战性的场景中提升性能。我们使用FaceForensics++、Celeb-DF、DeepfakeDetection和DFDC数据集进行评估，验证了方法的优越性。在内数据集评估中，模型取得AUC 99.49%和准确率97.57%。在跨数据集测试中，模型在未见的DeepFakeDetection和DFDC数据集上分别取得AUC 93.31%和81.25%。这些结果突显了基于跨注意力的特征融合在提升深度伪造视频检测鲁棒性方面的有效性。
英文摘要：
Deepfakes have emerged as a significant threat to digital media authenticity, increasing the need for advanced detection techniques that can identify subtle and time-dependent manipulations. CNNs are effective at capturing spatial artifacts and Transformers excel at modeling temporal inconsistencies. However, many existing CNN-Transformer models process spatial and temporal features independently. In particular, attention based methods often use independent attention mechanisms for spatial and temporal features and combine them using naive approaches like averaging, addition or concatenation, limiting the depth of spatio-temporal interaction. To address this challenge, we propose a unified CAST model that leverages cross-attention to effectively fuse spatial and temporal features in a more integrated manner. Our approach allows temporal features to dynamically attend to relevant spatial regions, enhancing the model's ability to detect fine-grained, time-evolving artifacts such as flickering eyes or warped lips. This design enables more precise localization and deeper contextual understanding, leading to improved performance across diverse and challenging scenarios. We evaluate the performance of our model using the FaceForensics++, Celeb-DF, DeepfakeDetection and Deepfake Detection Challenge (DFDC) datasets in both intra and cross dataset settings to affirm the superiority of our approach. Our model achieves strong performance with an Area Under the Curve (AUC) of 99.49% and an accuracy of 97.57% in intra-dataset evaluations. In cross-dataset testing, the model achieves AUC scores of 93.31% and 81.25% on the unseen DeepFakeDetection and DFDC datasets, respectively. These results highlight the effectiveness of cross-attention-based feature fusion in enhancing the robustness of deepfake video detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
