ChatGPT在音视频深度伪造检测中表现如何：ChatGPT、AI模型与人类感知的比较研究
How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception
AI总结：
本研究评估了ChatGPT在音视频深度伪造检测中的能力，发现结合领域知识与提示工程，它能有效解释跨模态伪影，并与最先进模型及人类感知进行了对比分析。
摘要：
涉及音视频篡改的多模态深度伪造日益威胁，因为它们难以用肉眼或基于单模态深度学习的伪造检测方法来发现。音视频取证模型虽然比单模态模型能力更强，但需要大型训练数据集，且训练和推理的计算成本高昂。此外，这些模型缺乏可解释性，且通常难以泛化到未见过的篡改。在本研究中，我们考察了大型语言模型（LLM）（即ChatGPT）的检测能力，以识别并解释音视频深度伪造内容中可能存在的视觉和听觉伪影及篡改。我们在基准多模态深度伪造数据集的视频上进行了大量实验，以评估ChatGPT的检测性能，并将其与最先进的多模态取证模型及人类的检测能力进行比较。实验结果证明了领域知识和提示工程对于使用LLM进行视频伪造检测任务的重要性。与基于端到端学习的方法不同，ChatGPT能够解释模态内部或跨模态之间可能存在的空间和时空伪影与不一致性。此外，我们还讨论了ChatGPT在多媒体取证任务中的局限性。
英文摘要：
Multimodal deepfakes involving audiovisual manipulations are a growing threat because they are difficult to detect with the naked eye or using unimodal deep learningbased forgery detection methods. Audiovisual forensic models, while more capable than unimodal models, require large training datasets and are computationally expensive for training and inference. Furthermore, these models lack interpretability and often do not generalize well to unseen manipulations. In this study, we examine the detection capabilities of a large language model (LLM) (i.e., ChatGPT) to identify and account for any possible visual and auditory artifacts and manipulations in audiovisual deepfake content. Extensive experiments are conducted on videos from a benchmark multimodal deepfake dataset to evaluate the detection performance of ChatGPT and compare it with the detection capabilities of state-of-the-art multimodal forensic models and humans. Experimental results demonstrate the importance of domain knowledge and prompt engineering for video forgery detection tasks using LLMs. Unlike approaches based on end-to-end learning, ChatGPT can account for spatial and spatiotemporal artifacts and inconsistencies that may exist within or across modalities. Additionally, we discuss the limitations of ChatGPT for multimedia forensic tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
