超越语义：通过通用物理描述符揭示合成检测的物理本质
Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection
AI总结：
本文提出通过通用物理描述符区分真实与AI生成图像，探索跨模态合成检测中的核心物理特征及多模态模型整合方法，实现高准确率检测。
摘要：
人工智能生成内容（AIGC）的快速发展模糊了真实与合成图像的边界，暴露了现有深度伪造检测器对特定生成模型过度拟合的局限性。本文针对两个关键问题：（1）哪些物理特征能稳定且稳健地在不同数据集和生成架构中区分AI生成图像？（2）这些客观像素级特征能否整合到CLIP等多模态模型中，以提升检测性能并缓解语言信息的不可靠性？为回答这些问题，我们全面探索了15种物理特征，涵盖超过20个数据集，由不同GAN和扩散模型生成。我们提出了一种新的特征选择算法，识别出包括拉普拉斯方差、索贝尔统计和残差噪声方差在内的五个核心物理特征，这些特征在所有测试数据集中表现出一致的判别能力。这些特征随后转换为文本编码值，并与语义描述相结合，以指导CLIP中的图像文本表示学习。大量实验表明，我们的方法在多个Genimage基准上实现了最先进的性能，例如在Wukong和SDv1.4数据集上达到近完美的准确率（99.8%）。通过将像素级真实性与语义理解相结合，本文开创了使用物理基础特征进行可信视觉语言模型的先河，并为减轻大多模态模型中的幻觉和文本不准确性开辟了新方向。
英文摘要：
The rapid advancement of AI generated content (AIGC) has blurred the boundaries between real and synthetic images, exposing the limitations of existing deepfake detectors that often overfit to specific generative models. This adaptability crisis calls for a fundamental reexamination of the intrinsic physical characteristics that distinguish natural from AI-generated images. In this paper, we address two critical research questions: (1) What physical features can stably and robustly discriminate AI generated images across diverse datasets and generative architectures? (2) Can these objective pixel-level features be integrated into multimodal models like CLIP to enhance detection performance while mitigating the unreliability of language-based information? To answer these questions, we conduct a comprehensive exploration of 15 physical features across more than 20 datasets generated by various GANs and diffusion models. We propose a novel feature selection algorithm that identifies five core physical features including Laplacian variance, Sobel statistics, and residual noise variance that exhibit consistent discriminative power across all tested datasets. These features are then converted into text encoded values and integrated with semantic captions to guide image text representation learning in CLIP. Extensive experiments demonstrate that our method achieves state-of-the-art performance on multiple Genimage benchmarks, with near-perfect accuracy (99.8%) on datasets such as Wukong and SDv1.4. By bridging pixel level authenticity with semantic understanding, this work pioneers the use of physically grounded features for trustworthy vision language modeling and opens new directions for mitigating hallucinations and textual inaccuracies in large multimodal models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
