面向可泛化AI生成图像检测的多模态条件信息瓶颈
Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection
AI总结：
针对现有CLIP基AI生成图像检测方法特征冗余、泛化性差的问题，提出文本引导框架InfoFD，通过TGCIB和DTO组件优化特征，在GenImage数据集与最新生成模型上实现优异泛化性能。
摘要：
尽管现有基于CLIP的AI生成图像检测方法已取得可观成效，但仍受严重的特征冗余限制，阻碍了其泛化能力。为解决该问题，将信息瓶颈网络引入此任务是一种直接的方案。然而，由于提示词本身具有多样性，仅依赖与图像对应的提示词会导致性能欠佳。本文提出一种多模态条件瓶颈网络，在减少特征冗余的同时增强CLIP提取特征的判别力，从而提升模型的泛化能力。我们首先开展了语义分析实验，观察到在CLIP特征空间中，任意文本特征与真实图像特征的余弦相似度低于其与生成图像特征的余弦相似度，我们将这一现象称为“偏差”。基于此，我们提出InfoFD——一种文本引导的AI生成图像检测框架。InfoFD包含两个核心组件：文本引导条件信息瓶颈（TGCIB）和动态文本正交化（DTO）。TGCIB通过同时以文本和类别模态为条件，提升所学表征的泛化性。DTO动态更新加权文本特征，在保留语义信息的同时利用全局“偏差”。我们的模型在GenImage数据集以及最新的生成模型上取得了优异的泛化性能。代码可在https://github.com/Ant0ny44/InfoFD获取。
英文摘要：
Although existing CLIP-based methods for detecting AI-generated images have achieved promising results, they are still limited by severe feature redundancy, which hinders their generalization ability. To address this issue, incorporating an information bottleneck network into the task presents a straightforward solution. However, relying solely on image-corresponding prompts results in suboptimal performance due to the inherent diversity of prompts. In this paper, we propose a multimodal conditional bottleneck network to reduce feature redundancy while enhancing the discriminative power of features extracted by CLIP, thereby improving the model's generalization ability. We begin with a semantic analysis experiment, where we observe that arbitrary text features exhibit lower cosine similarity with real image features than with fake image features in the CLIP feature space, a phenomenon we refer to as "bias". Therefore, we introduce InfoFD, a text-guided AI-generated image detection framework. InfoFD consists of two key components: the Text-Guided Conditional Information Bottleneck (TGCIB) and Dynamic Text Orthogonalization (DTO). TGCIB improves the generalizability of learned representations by conditioning on both text and class modalities. DTO dynamically updates weighted text features, preserving semantic information while leveraging the global "bias". Our model achieves exceptional generalization performance on the GenImage dataset and latest generative models. Our code is available at https://github.com/Ant0ny44/InfoFD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
