arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.11909cs.SDeess.AS
收藏

基于冻结wav2vec 2.0的混合专家融合方法用于伪造音频检测

Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0

University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所);Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心);AI Lab, Tencent(腾讯AI实验室)

Zhiyong Wang, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Xiaopeng Wang, Yuankun Xie, Xin Qi, Shuchen Shi, Yi Lu, Yukun Liu, Chenxing Li, Xuefei Liu, Guanjun Li

AI总结:

针对现有伪造音频检测中预训练模型微调耗时长、迭代慢的问题,提出冻结wav2vec 2.0下基于混合专家的特征融合方法,在两个基准数据集上取得了与需微调方法相当的竞争力性能。

摘要:

语音合成技术已对说话人验证系统构成严重威胁。目前,最有效的伪造音频检测方法均采用预训练模型,而整合预训练模型不同层的特征可进一步提升检测性能。然而,此前提出的大多数融合方法都需要对预训练模型进行微调,导致训练时间过长,在面对新型语音合成技术时阻碍了模型迭代。为解决这一问题,本文提出一种基于混合专家(Mixture of Experts)的特征融合方法,该方法在冻结预训练模型的前提下,以基于最后一层特征的门控网络为引导,从各层特征中提取并整合与伪造音频检测相关的特征。在ASVspoof2019和ASVspoof2021数据集上开展的实验表明,所提方法与需要微调的方法相比取得了具有竞争力的性能。

英文摘要:

Speech synthesis technology has posed a serious threat to speaker verification systems. Currently, the most effective fake audio detection methods utilize pretrained models, and integrating features from various layers of pretrained model further enhances detection performance. However, most of the previously proposed fusion methods require fine-tuning the pretrained models, resulting in excessively long training times and hindering model iteration when facing new speech synthesis technology. To address this issue, this paper proposes a feature fusion method based on the Mixture of Experts, which extracts and integrates features relevant to fake audio detection from layer features, guided by a gating network based on the last layer feature, while freezing the pretrained model. Experiments conducted on the ASVspoof2019 and ASVspoof2021 datasets demonstrate that the proposed method achieves competitive performance compared to those requiring fine-tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑