LaP-Forensics：用于深度伪造检测的潜在像素一致性引导的多模态推理
LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection
另 2 家 · 查看机构详情
- The Hong Kong Polytechnic University（香港理工大学）
- University College London（伦敦大学学院）
- Tsinghua University（清华大学）
- Sun Yat-sen University（中山大学）
- National University of Singapore（新加坡国立大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
研究针对深度伪造检测问题，提出LaP-Forensics多模态框架，利用基于重建的取证证据及结构化模型预测，经组相对策略优化，实现跨生成器检测和伪影定位，实验验证了残差流效用，但后处理下文本忠实性和可靠性有局限。
中文摘要 AI 辅助
近期生成模型能生成视觉伪影少的图像，削弱了仅依赖表面外观的检测器和解释。我们提出LaP-Forensics，一个用基于重建的取证证据增强RGB语义的多模态框架。冻结的Stable Diffusion DDIM反演-重建模型提供固定重建参考，其残差图衡量与该参考的局部兼容性。独立投影仪对RGB图像和残差图编码，结构化的Where-What-Why模型预测文本分析和伪影。微调后进行组相对策略优化（GRPO），其奖励结合掩码重叠与输出结构及证据-参考项。单独的图像级头部融合RGB和DDIM-残差类特征。实验表明在UniversalFakeDetect上有跨生成器检测能力，在官方SynthScars基准上有有竞争力的伪影定位能力。控制线索构建、反演范围、组件、奖励项和反事实分析支持了评估设置下残差流的效用，而后处理下的自由形式文本忠实性和可靠性仍是开放的局限性。
英文摘要
Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance. We present LaP-Forensics, a multimodal framework that augments RGB semantics with reconstruction-based forensic evidence. A frozen Stable Diffusion DDIM inversion-reconstruction model provides a fixed reconstruction reference, and its residual map measures local compatibility with that reference. Independent projectors encode the RGB image and residual map before a structured Where-What-Why model predicts a textual analysis and an artifact mask.Supervised fine-tuning is followed by Group Relative Policy Optimization (GRPO), whose reward combines mask overlap with output-structure and evidence-reference terms. These text-side terms encourage the model to refer to the consistency map but do not constitute a verifier of free-form textual truth. A separate image-level head fuses RGB and DDIM-residual class features. Experiments show cross-generator detection on UniversalFakeDetect and competitive artifact localization on the official SynthScars benchmark. Controlled cue-construction, inversion-horizon, component, reward-term, and counterfactual analyses support the utility of the residual stream under the evaluated settings, while free-form textual faithfulness and reliability under post-processing remain open limitations.
