基于空间-频率-光流多模态特征融合的AIGC视频检测

中文摘要 AI 辅助

生成式AI（如Sora、Hunyuan）的快速发展使得开发能够泛化于不断演进的合成技术的有效检测策略变得至关重要。本研究的动机源于对生成模型中一个基本挑战的观察：在外观与运动之间维持跨模态一致性的固有困难。为此，我们提出了一种用于AIGC视频伪造检测任务的多模态框架，名为基于交叉注意力的视频伪造检测器（CrossAtt-VFD），该框架基于空间-频率和光流特征的联合多视角分析。具体而言，我们引入了一个双分支架构，同时提取空间-频率特征和光流特征。该方法能够从互补的感知角度对视频进行建模。该过程的核心是一个专用的交叉注意力机制，它控制两种模态的对齐，并将跨模态不一致性转化为强有力的诊断信号。这种多模态策略有助于检测与场景视觉外观统计上不一致的运动。综合实验结果表明，我们的模型达到了94.22%的准确率、91.67%的精确率和96.25%的召回率，有效验证了多模态融合策略的优势。

英文摘要

The rapid evolution of generative AI (e.g., Sora, Hunyuan) makes it essential to develop effective detection strategies that can generalize across ever-evolving synthesis techniques. This study is motivated by the observation of a fundamental challenge in generative models: the inherent difficulty of maintaining cross-modal consistency between appearance and motion. To this end, we propose a multi-modal framework for AIGC video forgery detection tasks, named Cross-Attention based Video Forgery Detector (CrossAtt-VFD), based on joint multi-view analysis of content.Methodologically, we introduce a dual-branch architecture that simultaneously extracts spatial-frequency and optical-flow features.This approach enables the modeling of videos from complementary perceptual perspectives.The core of this process is a dedicated cross-attention mechanism, which governs the alignment of the two modalities and translates cross-modal inconsistencies into a potent diagnostic signal. This multi-modal strategy facilitates the detection of motion that is statistically inconsistent with the visual appearance of a scene. Comprehensive experimental results demonstrated that our model achieves an accuracy of 94.22%, a precision of 91.67 %,and a recall of 96.25 %, effectively verifying the advantages of the multi-modal fusion strategy.

发表机构

• Beihang University（北京航空航天大学）
• Nanchang University（南昌大学）
• DBAPPSecurity Co., Ltd. (DAS-Security)（杭州安恒信息技术股份有限公司）
• Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）

机构由 AI 辅助整理，请以论文原文为准。