VIGIL:基于部分的结构化推理用于通用深度伪造检测
VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection
AI总结:
VIGIL通过计划-检验流程,结合部分级证据和结构化推理,提升深度伪造检测的可解释性和泛化能力。
摘要:
多模态大语言模型(MLLMs)通过生成文本解释为深度伪造检测提供了有前景的可解释路径。然而,当前基于MLLM的方法将证据生成和 manipulation 定位合并为统一步骤,模糊了忠实观察与幻觉解释的边界,导致结论不可靠。基于此,我们提出VIGIL,一种受专家刑侦实践启发的基于部分的结构化刑侦框架:模型首先根据全局视觉线索计划需要检查的面部部分,然后使用独立来源的刑侦证据检查每个部分。阶段门控注入机制仅在检查期间提供部分级刑侦证据,确保部分选择由模型自身感知驱动而非外部信号偏见。我们进一步提出一种渐进的三阶段训练范式,其强化学习阶段采用部分感知奖励以强制解剖学有效性与证据-结论一致性。为实现严格的一般化评估,我们构建了OmniFake,一个五级分层基准,其中模型仅在三个基础生成器上训练,逐步测试到真实世界社交媒体数据。在OmniFake和跨数据集评估中,VIGIL在所有一般化级别上均优于专家检测器和同时期的MLLM方法。
英文摘要:
Multimodal large language models (MLLMs) offer a promising path toward interpretable deepfake detection by generating textual explanations. However, the reasoning process of current MLLM-based methods combines evidence generation and manipulation localization into a unified step. This combination blurs the boundary between faithful observations and hallucinated explanations, leading to unreliable conclusions. Building on this, we present VIGIL, a part-centric structured forensic framework inspired by expert forensic practice through a plan-then-examine pipeline: the model first plans which facial parts warrant inspection based on global visual cues, then examines each part with independently sourced forensic evidence. A stage-gated injection mechanism delivers part-level forensic evidence only during examination, ensuring that part selection remains driven by the model's own perception rather than biased by external signals. We further propose a progressive three-stage training paradigm whose reinforcement learning stage employs part-aware rewards to enforce anatomical validity and evidence--conclusion coherence. To enable rigorous generalizability evaluation, we construct OmniFake, a hierarchical 5-Level benchmark where the model, trained on only three foundational generators, is progressively tested up to in-the-wild social-media data. Extensive experiments on OmniFake and cross-dataset evaluations demonstrate that VIGIL consistently outperforms both expert detectors and concurrent MLLM-based methods across all generalizability levels.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。