arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.04324cs.SDcs.AIeess.AS
收藏

音频深度伪造检测发展到哪一步了?对生成模型与检测模型的系统性分析

Where are we in audio deepfake detection? A systematic analysis over generative and detection models

Fordham University(福特汉姆大学);IBM Research(IBM研究院)

Xiang Li, Pin-Yu Chen, Wenqi Wei

AI总结:

针对现有AI合成音频检测方法泛化性不足的问题,研究提出SONAR检测框架与基准,通过多组实验揭示了基础模型在检测泛化、跨语言适配等方面的优势及少样本微调的应用潜力。

摘要:

基于生成式人工智能(AI)技术的文本转语音(TTS)和语音转换(VC)技术近期取得进展,已能够生成高质量、逼真的类人音频。这给区分AI合成语音与真实人声带来了日益严峻的挑战,还可能引发人们对其被滥用于冒充、诈骗、传播虚假信息和欺诈的担忧。然而,现有的AI合成音频检测方法并未跟上发展步伐,往往无法在不同数据集间实现泛化。\n本文提出了SONAR,这是一个合成AI音频检测框架与基准,旨在为区分前沿AI合成听觉内容提供全面的评估体系。SONAR包含一个全新的评估数据集,数据来自9个不同的音频合成平台,涵盖头部TTS服务商和最先进的TTS模型。它是首个对传统检测系统和基于基础模型的检测系统进行统一AI音频检测基准测试的框架。\n通过大量实验,我们得到以下结论:(1)揭示了现有检测方法的局限性,证明基础模型展现出更强的泛化能力,这可能得益于其模型规模以及预训练数据的规模和质量。(2)语音基础模型展现出强劲的跨语言泛化能力,尽管仅在英语语音数据上进行微调,仍能在不同语言中保持优异性能。这一发现也表明,音频深度伪造检测的主要挑战更多与合成音频的逼真度和质量相关,而非语言特定特征。(3)探究了少样本微调在提升泛化性方面的效果与效率,凸显了其在定制化应用中的潜力,例如针对特定实体或个人的个性化检测系统。

英文摘要:

Recent advances in Text-to-Speech (TTS) and Voice-Conversion (VC) using generative Artificial Intelligence (AI) technology have made it possible to generate high-quality and realistic human-like audio. This poses growing challenges in distinguishing AI-synthesized speech from the genuine human voice and could raise concerns about misuse for impersonation, fraud, spreading misinformation, and scams. However, existing detection methods for AI-synthesized audio have not kept pace and often fail to generalize across diverse datasets. In this paper, we introduce SONAR, a synthetic AI-Audio Detection Framework and Benchmark, aiming to provide a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content. SONAR includes a novel evaluation dataset sourced from 9 diverse audio synthesis platforms, including leading TTS providers and state-of-the-art TTS models. It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based detection systems. Through extensive experiments, (1) we reveal the limitations of existing detection methods and demonstrate that foundation models exhibit stronger generalization capabilities, likely due to their model size and the scale and quality of pretraining data. (2) Speech foundation models demonstrate robust cross-lingual generalization capabilities, maintaining strong performance across diverse languages despite being fine-tuned solely on English speech data. This finding also suggests that the primary challenges in audio deepfake detection are more closely tied to the realism and quality of synthetic audio rather than language-specific characteristics. (3) We explore the effectiveness and efficiency of few-shot fine-tuning in improving generalization, highlighting its potential for tailored applications, such as personalized detection systems for specific entities or individuals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑