多说话者对话音频深度伪造：分类、数据集和初步研究
Multi-Speaker Conversational Audio Deepfake: Taxonomy, Dataset and Pilot Study
AI总结：
本文提出多说话人对话音频深度伪造的分类方法，并引入MsCADD数据集，评估了三种基线模型在检测合成语音方面的性能，揭示了多说话人深度伪造检测的挑战。
摘要：
文本到语音（TTS）技术的快速发展使音频深度伪造变得更加逼真和易于获取，引发了重大安全和信任问题。尽管现有研究主要集中在检测单说话人音频深度伪造，但现实世界中涉及多说话人对话设置的恶意应用也正在成为一个主要的未探索威胁。为了解决这一差距，我们提出了多说话人对话音频深度伪造的概念分类，区分部分篡改（一个或多个说话人被修改）和完整篡改（整个对话被合成）。作为第一步，我们引入了一个新的多说话人对话音频深度伪造数据集（MsCADD），包含2,830个音频片段，包含真实和完全合成的双说话人对话，使用VITS和基于NotebookLM的SoundStorm模型生成，以模拟自然对话，包括说话人性别和对话自发性的变化。MsCADD仅限于文本到语音（TTS）类型的深度伪造。我们在此数据集上基准测试了三个神经网络基线模型；LFCC-LCNN、RawNet2和Wav2Vec 2.0，并报告了F1分数、准确率、真阳性率（TPR）和真阴性率（TNR）的性能。结果表明，这些基线模型提供了有用的基准，但结果也表明在可靠检测合成声音方面，多说话人深度伪造研究存在显著差距。我们的数据集和基准为未来研究深度伪造检测在对话场景中的研究提供了基础，这是一个高度未探索的研究领域，但也是音频设置中可信信息的主要威胁领域。MsCADD数据集已公开供研究社区用于可重复性和基准测试。
英文摘要：
The rapid advances in text-to-speech (TTS) technologies have made audio deepfakes increasingly realistic and accessible, raising significant security and trust concerns. While existing research has largely focused on detecting single-speaker audio deepfakes, real-world malicious applications with multi-speaker conversational settings is also emerging as a major underexplored threat. To address this gap, we propose a conceptual taxonomy of multi-speaker conversational audio deepfakes, distinguishing between partial manipulations (one or multiple speakers altered) and full manipulations (entire conversations synthesized). As a first step, we introduce a new Multi-speaker Conversational Audio Deepfakes Dataset (MsCADD) of 2,830 audio clips containing real and fully synthetic two-speaker conversations, generated using VITS and SoundStorm-based NotebookLM models to simulate natural dialogue with variations in speaker gender, and conversational spontaneity. MsCADD is limited to text-to-speech (TTS) types of deepfake. We benchmark three neural baseline models; LFCC-LCNN, RawNet2, and Wav2Vec 2.0 on this dataset and report performance in terms of F1 score, accuracy, true positive rate (TPR), and true negative rate (TNR). Results show that these baseline models provided a useful benchmark, however, the results also highlight that there is a significant gap in multi-speaker deepfake research in reliably detecting synthetic voices under varied conversational dynamics. Our dataset and benchmarks provide a foundation for future research on deepfake detection in conversational scenarios, which is a highly underexplored area of research but also a major area of threat to trustworthy information in audio settings. The MsCADD dataset is publicly available to support reproducibility and benchmarking by the research community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
