arXiv 2606.158802026-06-16cs.CVcs.AI

深度残差注入：多模态大语言模型的全频谱取证信号感知

Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models

National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；University of Electronic Science and Technology of China（电子科技大学）；University of California, Berkeley（加州大学伯克利分校）

Kaiqing Lin, Zhiyuan Yan, Ruoxin Chen, Ke-Yue Zhang, Yue Zhou, Caiyong Piao, Bin Li, Taiping Yao, Bo Wang, Youchang Xiao, Shouhong Ding

备注：: Accepted at ICML 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多模态大语言模型在取证中难以同时保留语义知识和捕获低级生成器伪影的问题，提出Deep-VRM方法，通过将伪影特定视觉信号作为残差路径注入中间层，实现全频谱信号感知，达到鲁棒检测性能。

摘要：

多模态大语言模型（MLLMs）因其强大的语义理解能力，越来越多地被应用于取证领域。随着AI生成图像变得逼真，仅凭语义层面的不一致往往不足以进行可靠检测。这引发了一个关键问题：MLLMs能否实现全频谱取证信号感知，即在不牺牲预训练语义知识的情况下捕获低级生成器伪影。我们进一步对MLLMs中的取证信号感知进行了逐层分析，表明语义信息主要在早期到中间层形成，而直接微调学习伪影会破坏这些语义表示。基于这一发现，我们提出了深度视觉残差MLLM（Deep-VRM），以保留早期语义处理，同时将伪影特定的视觉信号作为残差路径注入中间层，在此与语义标记表示融合，并通过后续可训练层传播。这使得后续层能够联合建模语义推理和信号级取证线索，令人惊讶的是，模型学会了根据输入自适应地利用不同级别的取证信号，实现了鲁棒且可泛化的检测性能。大量实验表明，我们的方法在大多数基准测试中达到了最先进水平。代码和数据可在https://github.com/KQL11/Deep-VRM获取。

英文摘要：

Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding. As AI-generated images become realistic, semantic-level inconsistencies alone are often insufficient for reliable detection. This motivates a critical question: whether MLLMs can achieve full-spectrum forensic signal perception, i.e., capturing low-level generator artifacts without sacrificing pre-trained semantic knowledge. We further perform a layer-wise analysis of forensic signal perception in MLLMs, showing that semantic information is primarily formed in the early-to-middle layers, whereas direct fine-tuning for artifact learning disrupts these semantic representations. Based on this insight, we propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer, where they are fused with semantic token representations and propagated through subsequent trainable layers. This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input, achieving robust and generalizable detection performance. Extensive experiments show that our method achieves state-of-the-art across most benchmarks. The code and data are available at https://github.com/KQL11/Deep-VRM.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。