arXiv 2501.082382026-06-05cs.SDeess.AS

CodecFake+: 基于编解码器的重合成数据作为检测CodecFake语音的代理

CodecFake+: Codec-Based Resynthesized Data as a Proxy for Detecting CodecFake Speech

Graduate Institute of Communication Engineering, National Taiwan University（国家交通大学通信工程研究院）；Department of Computer Science and Information Engineering, National Taiwan University（国家交通大学计算机科学与信息工程系）；Center for Language and Speech Processing at Johns Hopkins University（约翰霍普金斯大学语言与语音处理中心）；Department of Electrical Engineering, National Taiwan University（国家交通大学电子工程系）；Research Center for Information Technology Innovation, Academia Sinica（学术院信息技术创新研究中心）；NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)（国家交通大学人工智能研究中心）

Xuanjun Chen, Jiawei Du, Haibin Wu, Lin Zhang, I-Ming Lin, I-Hsiang Chiu, Wenze Ren, Yuan Tseng, Yu Tsao, Jyh-Shing Roger Jang, Hung-yi Lee

备注：: Accepted by TASLP 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对新兴的CodecFake深度伪造语音检测挑战，提出大规模数据集CodecFake+，包含31种开源编解码器重合成训练数据和17种先进CoSG模型网络数据，并建立编解码器分类体系，验证了重合成语音作为训练数据的有效性。

摘要：

随着神经音频编解码器的快速发展，基于编解码器的语音生成（CoSG）系统变得非常强大。不幸的是，CoSG也使得创建高度逼真的深度伪造语音成为可能，更容易模仿个人声音并传播错误信息。我们将这种由CoSG系统生成的新兴深度伪造语音称为CodecFake。检测这种CodecFake是一个紧迫的挑战，然而现有系统大多主要关注检测由传统语音合成模型生成的伪造语音。在本文中，我们介绍了CodecFake+，一个旨在推进CodecFake检测的大规模数据集。据我们所知，CodecFake+是包含最多样化编解码器架构的最大数据集。训练集通过使用31个公开可用的开源编解码器模型进行重合成生成，而评估集包括来自17个先进CoSG模型的网络数据。我们还提出了一个全面的分类体系，根据编解码器的根组件：向量量化器、辅助目标和解码器类型对其进行分类。我们提出的数据集和分类体系使得能够在多个层面进行详细分析，以辨别成功检测CodecFake的关键因素。在单个编解码器层面，我们验证了使用编解码器重合成语音（CoRS）作为训练数据用于大规模CodecFake检测的有效性。在分类体系层面，我们表明当重合成模型包含解缠辅助目标或频域解码器时，检测性能最强。此外，从使用所有CoRS训练数据的角度，我们表明我们提出的分类体系可用于选择更好的训练数据以提高检测性能。总体而言，我们期望CodecFake+将成为通用和细粒度探索的重要资源，以开发更好的针对CodecFake的反欺骗模型。

英文摘要：

With the rapid advancement of neural audio codecs, codec-based speech generation (CoSG) systems have become highly powerful. Unfortunately, CoSG also enables the creation of highly realistic deepfake speech, making it easier to mimic an individual's voice and spread misinformation. We refer to this emerging deepfake speech generated by CoSG systems as CodecFake. Detecting such CodecFake is an urgent challenge, yet most existing systems primarily focus on detecting fake speech generated by traditional speech synthesis models. In this paper, we introduce CodecFake+, a large-scale dataset designed to advance CodecFake detection. To our knowledge, CodecFake+ is the largest dataset encompassing the most diverse range of codec architectures. The training set is generated through re-synthesis using 31 publicly available open-source codec models, while the evaluation set includes web-sourced data from 17 advanced CoSG models. We also propose a comprehensive taxonomy that categorizes codecs by their root components: vector quantizer, auxiliary objectives, and decoder types. Our proposed dataset and taxonomy enable detailed analysis at multiple levels to discern the key factors for successful CodecFake detection. At the individual codec level, we validate the effectiveness of using codec re-synthesized speech (CoRS) as training data for large-scale CodecFake detection. At the taxonomy level, we show that detection performance is strongest when the re-synthesis model incorporates disentanglement auxiliary objectives or a frequency-domain decoder. Furthermore, from the perspective of using all the CoRS training data, we show that our proposed taxonomy can be used to select better training data for improving detection performance. Overall, we envision that CodecFake+ will be a valuable resource for both general and fine-grained exploration to develop better anti-spoofing models against CodecFake.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。