TranX-Adapter: 在MLLMs中弥合artifact与语义以实现鲁棒的AI生成图像检测
TranX-Adapter: Bridging Artifacts and Semantics within MLLMs for Robust AI-generated Image Detection
AI总结：
TranX-Adapter通过引入任务感知的最优传输融合和X-Fusion机制，在MLLMs中提升AI生成图像检测的鲁棒性和准确性。
摘要：
AI生成图像（AIGI）技术的快速进步使得高度逼真的合成成为可能，威胁着公共信息的完整性和安全。最近的研究表明，在多模态大语言模型（MLLMs）中结合纹理层面的artifact特征和语义特征可以增强其AIGI检测能力。然而，我们的初步分析表明，artifact特征表现出高内在特征相似性，导致softmax操作后几乎均匀的注意力图。这种现象导致注意力稀释，从而阻碍了语义和artifact特征之间的有效融合。为克服这一限制，我们提出了一种轻量级融合适配器TranX-Adapter，它集成了一个任务感知的最优传输融合，利用artifact和语义预测概率之间的Jensen-Shannon散度作为成本矩阵，将artifact信息转移到语义特征中，并且一个X-Fusion，利用交叉注意力将语义信息转移到artifact特征中。在标准AIGI检测基准上对几种先进的MLLMs进行实验，结果显示，我们的TranX-Adapter带来了一致且显著的改进（最高+6%准确率）。
英文摘要：
Rapid advances in AI-generated image (AIGI) technology enable highly realistic synthesis, threatening public information integrity and security. Recent studies have demonstrated that incorporating texture-level artifact features alongside semantic features into multimodal large language models (MLLMs) can enhance their AIGI detection capability. However, our preliminary analyses reveal that artifact features exhibit high intra-feature similarity, leading to an almost uniform attention map after the softmax operation. This phenomenon causes attention dilution, thereby hindering effective fusion between semantic and artifact features. To overcome this limitation, we propose a lightweight fusion adapter, TranX-Adapter, which integrates a Task-aware Optimal-Transport Fusion that leverages the Jensen-Shannon divergence between artifact and semantic prediction probabilities as a cost matrix to transfer artifact information into semantic features, and an X-Fusion that employs cross-attention to transfer semantic information into artifact features. Experiments on standard AIGI detection benchmarks upon several advanced MLLMs, show that our TranX-Adapter brings consistent and significant improvements (up to +6% accuracy).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
