音频深度伪造检测：基于时间一致性分析
Audio Deepfake Detection Using Temporal Coherence Analysis
- University of California, Berkeley（加州大学伯克利分校）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出基于CLAP嵌入的时间一致性分析框架，通过余弦相似度与统计特征训练轻量集成分类器，有效检测语音和音乐深度伪造，并发现特征-标签反转与语音-音乐方向反转现象。
AI中文摘要：
AI生成音频（即所谓的“深度伪造”音频）的激增对信息完整性构成了重大威胁，从语音克隆欺诈到合成音乐版权纠纷。我们提出了一种基于对比语言-音频预训练（CLAP）嵌入的时间一致性分析框架，该框架涵盖语音、器乐和带人声的音乐。通过计算音频片段嵌入之间的成对余弦相似度，并从所得分布中提取统计特征，我们训练了轻量级集成分类器，能够可靠地区分真实音频与合成音频。我们的工作为常见的深度学习方法提供了一种可解释、计算高效的替代方案，同时在语音和音乐领域仍能达到有竞争力的性能。此外，我们揭示了关于音频深度伪造的两个显著实证发现：（1）特征-标签反转现象，其中29个统计特征中有21个在训练和实际部署之间逆转了其判别方向；（2）语音-音乐方向反转，其中熵对语音和音乐深度伪造的判别方向相反。
英文摘要：
The proliferation of AI-generated audio (so-called "deepfake" audio) poses significant threats to information integrity, from voice cloning fraud to synthetic music copyright disputes. We present a temporal coherence analysis framework built upon Contrastive Language-Audio Pretraining (CLAP) embeddings that spans speech, instrumental music, and music with vocals. By computing pairwise cosine similarities between audio segment embeddings and extracting statistical features from the resulting distributions, we train lightweight ensemble classifiers that reliably distinguish authentic from synthetic audio. Our work provides an interpretable, computationally efficient alternative to common deep learning methods while still achieving competitive performance across speech and music domains. Further, we reveal two notable empirical findings about audio deepfakes: (1) a feature-label inversion phenomenon in which 21 of 29 statistical features reverse their discriminative direction between training and in-the-wild deployment, and (2) a speech--music direction reversal in which entropy discriminates in opposite directions for speech and music deepfakes.
