发表机构
Shenzhen University; Shandong Artificial Intelligence Institute; Qilu University of Technology (Shandong Academy of Sciences); Tianjin University of Technology（深圳大学; 山东省人工智能研究院; 齐鲁工业大学（山东省科学院）; 天津理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文构建人口统计平衡的深度伪造检测基准，提出FairForensics视觉-语言模型，通过表情感知与人口统计正则化实现通用公平检测，在基准上达到泛化与公平性的SOTA。
AI 中文摘要
公平深度伪造检测（FDD）的挑战正受到越来越多的关注。现有的公平增强型检测器往往存在对未见过的伪造操作泛化能力不佳、以及不同人口统计群体间公平性不足的问题。它们通常在人口统计分布不平衡的数据集上开发和评估，导致对少数群体的预测存在偏差。在本文中，我们构建了一个新颖的人口统计平衡FDD基准，用于在平衡和不平衡人口场景下训练和评估检测器的公平性。此外，我们提出了一种新颖的表情与人口统计感知视觉-语言模型，名为FairForensics，用于通用公平深度伪造检测。FairForensics执行面部伪造泛化增强和人口统计感知公平正则化。在面部伪造泛化增强阶段，基于对原始和伪造表情向量之间存在显著分布差异的新观察，我们设计了一个表情编码器以捕获高级表情引导的伪造模式，以及一个表情感知视觉编码器，该编码器整合全局外观和表情伪造特征，同时使用身份感知补丁扰动模块减轻身份偏差。在人口统计感知公平正则化阶段，我们提出了一个人口统计引导语言编码器，以提取人口感知全局语言嵌入，该嵌入通过视觉-语言对齐促进伪造特征与人口统计信息的解耦。我们设计了一个人口感知原型公平目标，以在人口统计子群体间强制执行类间可分性和类内对齐。在我们的平衡人口统计基准上进行的大量实验表明，我们的方法在泛化性和公平性方面达到了当前最先进水平。
英文摘要
The challenge of fair deepfake detection (FDD) has attracted increasing attention. Existing fairness-enhanced detectors often suffer from suboptimal generalization to unseen manipulations and fairness across demographic groups. They are typically developed and evaluated on demographically imbalanced distributions, resulting in biased predictions toward minority groups. In this paper, we construct a novel demographically balanced FDD benchmark to train and evaluate the fairness of detectors under both balanced and imbalanced population scenarios. Additionally, we introduce a novel expression and demographic perceptual vision-language model, termed FairForensics, for generalizable fair deepfake detection. FairForensics conducts face forgery generalization enhancement and demographic-aware fairness regularization. During face forgery generalization enhancement, built upon the novel observation of significant distribution differences between pristine and forged expression vectors, we design an expression encoder to capture high-level expression-guided forgery patterns, and an expression-perceptual visual encoder that integrates global appearance and expression forgery features while mitigating identity bias using an identity-aware patch perturbation module. Under demographic-aware fairness regularization, we propose a demographic-guided language encoder to extract population-aware global language embeddings, which boosts the decoupling of forgery features from demographic information via vision-language alignment. We devise a population-aware prototype fairness objective to enforce both inter-class separability and intra-class alignment across demographic subgroups. Extensive experiments on our balanced demographic benchmark show that our method achieves the state-of-the-art in terms of generalization and fairness.
