arXiv 2406.081122024-06-12cs.SDcs.AIeess.AS

Codecfake：一个用于检测基于LLM的深度伪造音频的初始数据集

Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Information and Communication Engineering, Communication University of China（中国传媒大学信息与通信工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Beijing National Research Center for Information Science and Technology, Tsinghua University（清华大学北京信息科学与技术国家研究中心）；Shanghai Polytechnic University（上海第二工业大学）

Yi Lu, Yuankun Xie, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Zhiyong Wang, Xin Qi, Xuefei Liu, Yongwei Li, Yukun Liu, Xiaopeng Wang, Shuchen Shi

备注：: Accepted by INTERSPEECH 2024. arXiv admin note: substantial text overlap with arXiv:2405.04880

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对基于LLM的深度伪造音频绕过声码器伪影导致现有检测失效的问题，提出由七种神经编解码方法生成的Codecfake数据集，并证明编解码器训练的ADD模型可将平均等错误率降低41.406%。

摘要：

随着基于大语言模型（LLM）的深度伪造音频的激增，迫切需要有效的检测方法。以往的深度伪造音频生成方法通常涉及多步生成过程，最后一步使用声码器从手工特征预测波形。然而，基于LLM的音频是在端到端生成过程中直接从离散神经编解码器生成的，跳过了最后的声码器处理步骤。这对当前基于声码器伪影的音频深度伪造检测（ADD）模型构成了重大挑战。为了有效检测基于LLM的深度伪造音频，我们聚焦于生成过程的核心，即从神经编解码器到波形的转换。我们提出了Codecfake数据集，该数据集由七种代表性的神经编解码方法生成。实验结果表明，在Codecfake测试集上，与基于声码器训练的ADD模型相比，基于编解码器训练的ADD模型的平均等错误率降低了41.406%。

英文摘要：

With the proliferation of Large Language Model (LLM) based deepfake audio, there is an urgent need for effective detection methods. Previous deepfake audio generation methods typically involve a multi-step generation process, with the final step using a vocoder to predict the waveform from handcrafted features. However, LLM-based audio is directly generated from discrete neural codecs in an end-to-end generation process, skipping the final step of vocoder processing. This poses a significant challenge for current audio deepfake detection (ADD) models based on vocoder artifacts. To effectively detect LLM-based deepfake audio, we focus on the core of the generation process, the conversion from neural codec to waveform. We propose Codecfake dataset, which is generated by seven representative neural codec methods. Experiment results show that codec-trained ADD models exhibit a 41.406% reduction in average equal error rate compared to vocoder-trained ADD models on the Codecfake test set.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。