EVAS:通过视听协同和引导边界校准实现高效多模态时间伪造定位
EVAS: Efficient Multimodal Temporal Forgery Localization via Audio-Visual Synergy and Steered Boundary Calibration
AI总结:
针对人工智能生成内容带来的多模态取证挑战,提出EVAS框架,利用多阶段视听协同机制和边界感知细化策略,结合解耦训练范式与轻量级网络,有效定位长视频中稀疏分布的伪造片段。
摘要:
人工智能生成内容的迅速扩散需要可靠的多模态取证。除了视频级别的二元分类,在长视频中精确定位稀疏分布的伪造片段仍然是一项关键挑战。当操作被巧妙嵌入且跨模态信号微弱且在时间上分散时,这项任务尤其困难。为了应对这些挑战,我们提出了EVAS,一个用于时间伪造定位的端到端多模态框架。其核心是一个多阶段视听协同机制,促进渐进式跨模态交互,以学习深度多模态取证表示并捕获稀疏操作的高阶语义痕迹。此外,我们引入了一种边界感知细化策略来实现引导边界校准。通过纳入无效帧掩码,该策略抑制了模糊区域并锐化了过渡预测。我们采用带有辅助头的解耦训练范式,将表示学习与推理目标分离,增强模型的泛化能力和稳定性。此外,还引入了一个轻量级的沙漏FFN来减少计算开销。广泛的实验表明,EVAS在三个基准数据集上实现了领先的平均定位精度和平均召回率,验证了其在细粒度时间伪造定位方面的有效性。
英文摘要:
The rapid proliferation of artificial intelligence-generated content necessitates reliable multimodal forensics. Beyond video-level binary classification, precisely localizing sparsely distributed forged segments in long-form videos remains a critical challenge. This task is particularly difficult when manipulations are subtly embedded and cross-modal signals are weak and temporally diffuse. To address these challenges, we propose EVAS, an end-to-end multimodal framework for temporal forgery localization. At its core, a Multi-Stage Audio-Visual Synergy mechanism facilitates progressive cross-modal interaction to learn deep multimodal forensic representations and capture high-order semantic traces of sparse manipulations. Furthermore, we introduce a Boundary-Aware Refinement strategy to achieve steered boundary calibration. By incorporating invalid-frame masking, this strategy suppresses ambiguous regions and sharpens transition predictions. We adopt a decoupled training paradigm with auxiliary heads to disentangle representation learning from inference objectives, enhancing model generalization and stability. Additionally, a lightweight HourglassFFN is incorporated to reduce computational overhead. Extensive experiments demonstrate that EVAS achieves state-of-the-art average localization accuracy and average recall across three benchmark datasets, validating its effectiveness for fine-grained temporal forgery localization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。