arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2608.08009cs.CVcs.AI

基于证据的取证推理:检测与定位多模态媒体篡改

Evidence-Grounded Forensic Reasoning for Detecting and Grounding Multi-Modal Media Manipulation

Yichun Yeh, Yiheng Li, Xiaobo Hu, Zhen Lei, Yang Yang

AI总结:

本文针对多模态媒体篡改检测与定位问题,提出基于证据的取证推理框架,结合锚定-验证推理链、可验证奖励系统与模态解耦优势路由机制,实现最优性能与可解释性的统一。

AI中文摘要:

虚假新闻越来越依赖跨模态图文伪造,使得透明且可验证的推理链成为检测与定位多模态媒体篡改(DGM4)的迫切需求。现有方法仅给出黑盒检测结果,无任何决策依据,限制了其在取证实践中的可靠性。多模态大语言模型(MLLM)为可解释性提供了自然途径,但将其应用于DGM4存在两大难点:一是模型倾向于生成与预测证据位置脱节的解释,产生无法验证的归因;二是强制证据与结论的一致性需要主动优化,而统一训练信号无法区分定位标记与分类标记,导致多头联合训练不可靠。本文提出一种基于证据的取证推理(EFR)框架的多模态篡改检测器。EFR引入锚定-验证推理链,在跨模态对比前执行模态隔离感知,将结论坐标作为明确锚点,下游证据必须在空间上与该锚点对应;可验证奖励系统在训练期间强制证据与结论的一致性,而模态解耦优势(MDA)路由机制缓解了预测任务间的信用分配错误。实验表明,EFR达到了当前最优性能,同时生成结构化的取证推理记录,将解释与证据明确绑定。

英文摘要:

Fake news increasingly relies on cross-modal image-text forgeries, making transparent and verifiable reasoning chains an urgent need for Detecting and Grounding Multi-Modal Media Manipulation (DGM4). Existing methods produce black-box detection results without any decision rationale, limiting their reliability in forensic practice. Multi-modal Large Language Models (MLLMs) offer a natural path toward explainability, but applying them to DGM4 raises two difficulties. First, models tend to generate explanations disconnected from predicted evidence locations, producing unverified attribution. Second, enforcing evidence-conclusion consistency requires active optimization, yet uniform training signals fail to distinguish localization tokens from classification tokens, making multi-head joint training unreliable. We propose a multi-modal manipulation detector based on an Evidence-Grounded Forensic Reasoning (EFR) framework. EFR introduces an Anchor-and-Verify reasoning chain that enforces modality-isolated perception before cross-modal comparison, with conclusion coordinates as explicit anchors to which downstream evidence must spatially correspond. A verifiable reward system then enforces evidence-conclusion consistency during training, while a Modality-Decoupled Advantage (MDA) routing mechanism mitigats credit misassignment across prediction tasks. Experiments show that EFR achieves state-of-the-art performance while producing structured forensic reasoning records that explicitly bind explanations to evidence.

补充信息

↑