深度伪造检测中的“小事精明、大事糊涂”现象
Penny-Wise and Pound-Foolish in Deepfake Detection
AI总结：
针对深度伪造检测中现有方法过度拟合单数据集、泛化性与知识保留差的“小事精明、大事糊涂”问题，提出基于预训练视觉-语言模型的PoundNet框架，通过可学习提示和平衡目标实现性能显著提升。
摘要：
深度伪造技术的扩散引发了人们对其在各领域潜在滥用的严重担忧，迫切需要鲁棒的检测方法。尽管已有进展，但当前许多方法以牺牲长期有效性为代价，优先追求短期收益。本文批判了仅在单个深度伪造数据集上以“小事精明”的目标微调预训练模型的过度专门化方法，该方法忽视了泛化性与知识保留的“大事”层面平衡。为解决这一“小事精明、大事糊涂”的问题，我们提出了一种新型学习框架PoundNet，用于在预训练视觉-语言模型上实现深度伪造检测的泛化。PoundNet结合了可学习提示设计与平衡目标，在增强下游任务（深度伪造检测）泛化性的同时，保留上游任务（目标分类）的广泛知识。我们遵循文献中的常见做法，在标准单个深度伪造数据集上训练PoundNet，随后在10个公开大规模深度伪造数据集上采用5项主要评估指标测试其性能——据我们所知，这是用于评估深度伪造检测模型泛化能力的最大基准测试集。综合基准评估表明，所提出的PoundNet大幅缓解了“小事精明、大事糊涂”的问题：与现有最优方法相比，深度伪造检测性能显著提升19%，同时在目标分类任务上保持63%的优异性能，而其他深度伪造检测模型在该任务上往往表现不佳。代码与数据已在https://github.com/iamwangyabin/PoundNet开源。
英文摘要：
The diffusion of deepfake technologies has sparked serious concerns about its potential misuse across various domains, prompting the urgent need for robust detection methods. Despite advancement, many current approaches prioritize short-term gains at expense of long-term effectiveness. This paper critiques the overly specialized approach of fine-tuning pre-trained models solely with a penny-wise objective on a single deepfake dataset, while disregarding the pound-wise balance for generalization and knowledge retention. To address this "Penny-Wise and Pound-Foolish" issue, we propose a novel learning framework (PoundNet) for generalization of deepfake detection on a pre-trained vision-language model. PoundNet incorporates a learnable prompt design and a balanced objective to preserve broad knowledge from upstream tasks (object classification) while enhancing generalization for downstream tasks (deepfake detection). We train PoundNet on a standard single deepfake dataset, following common practice in the literature. We then evaluate its performance across 10 public large-scale deepfake datasets with 5 main evaluation metrics-forming the largest benchmark test set for assessing the generalization ability of deepfake detection models, to our knowledge. The comprehensive benchmark evaluation demonstrates the proposed PoundNet is significantly less "Penny-Wise and Pound-Foolish", achieving a remarkable improvement of 19% in deepfake detection performance compared to state-of-the-art methods, while maintaining a strong performance of 63% on object classification tasks, where other deepfake detection models tend to be ineffective. Code and data are open-sourced at https://github.com/iamwangyabin/PoundNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
