Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所);Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心);AI Lab, Tencent(腾讯AI实验室)
语音合成技术已对说话人验证系统构成严重威胁。目前,最有效的伪造音频检测方法均采用预训练模型,而整合预训练模型不同层的特征可进一步提升检测性能。然而,此前提出的大多数融合方法都需要对预训练模型进行微调,导致训练时间过长,在面对新型语音合成技术时阻碍了模型迭代。为解决这一问题,本文提出一种基于混合专家(Mixture of Experts)的特征融合方法,该方法在冻结预训练模型的前提下,以基于最后一层特征的门控网络为引导,从各层特征中提取并整合与伪造音频检测相关的特征。在ASVspoof2019和ASVspoof2021数据集上开展的实验表明,所提方法与需要微调的方法相比取得了具有竞争力的性能。
英文摘要:
Speech synthesis technology has posed a serious threat to speaker verification systems.
Currently, the most effective fake audio detection methods utilize pretrained models, and integrating features from various layers of pretrained model further enhances detection performance.
However, most of the previously proposed fusion methods require fine-tuning the pretrained models, resulting in excessively long training times and hindering model iteration when facing new speech synthesis technology.
To address this issue, this paper proposes a feature fusion method based on the Mixture of Experts, which extracts and integrates features relevant to fake audio detection from layer features, guided by a gating network based on the last layer feature, while freezing the pretrained model.
Experiments conducted on the ASVspoof2019 and ASVspoof2021 datasets demonstrate that the proposed method achieves competitive performance compared to those requiring fine-tuning.
EmoFake: An Initial Dataset for Emotion Fake Audio Detection
School of Artificial Intelligence, Hebei University of Technology(河北工业大学人工智能学院);National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);Department of Automation, Tsinghua University(清华大学自动化系)
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心);University of Science and Technology of China(中国科学技术大学)
Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学)
TO-Rawnet: Improving RawNet with TCN and Orthogonal Regularization for Fake Audio Detection
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学)
A Noval Feature via Color Quantisation for Fake Audio Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Information and Communication Engineering, Communication University of China(中国传媒大学信息与通信工程学院);Department of Automation, Tsinghua University(清华大学自动化系);Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
Johns Hopkins University(约翰霍普金斯大学);Brno University of Technology(布拉格技术大学);National Institute of Informatics(国家信息研究所);National University of Singapore(新加坡国立大学);University of Rochester(罗切斯特大学);Nanyang Technological University(南洋理工大学);Nanjing University(南京大学);WeNet Opensource Community(WeNet开源社区);EURECOM
Half-Truth: A Partially Fake Audio Detection Dataset
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学)
本文构建 HAD 部分伪造音频检测数据集,采用最新语音合成技术仅替换语句中的少数词语,支持伪造检测与篡改区域定位,并通过基准实验证明其比完全伪造音频更具挑战性。
Genuine-Focused Learning using Mask AutoEncoder for Generalized Fake Audio Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Tsinghua University(清华大学);School of Information and Communication Engineering, Communication University of China(中国传媒大学信息与通信工程学院);Shanghai Polytechnic University(上海第二工业大学)
SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);Anhui University(安徽大学)
ADD 2023: the Second Audio Deepfake Detection Challenge
Peoples' Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学(RUDN大学));Joint Institute for Nuclear Research(联合核研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Skövde(舍夫德大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong(香港中文大学)
Adaptive Fake Audio Detection with Low-Rank Model Squeezing
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Jiaotong University(北京交通大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学)
ADD 2022: the First Audio Deep Synthesis Detection Challenge
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学)
Generalized Fake Audio Detection via Deep Stable Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, Chinese Academy of Sciences(中国科学院人工智能学院);Tsinghua University(清华大学);Communication University of China(中国传媒大学);Shanghai Polytechnic University(上海第二工业大学)
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心)
Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy
Communication University of China(中国传媒大学);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学)
RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Tsinghua University(清华大学)
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
EURECOM(欧洲通信研究院);National Institute of Informatics(国立信息学研究所);University of Eastern Finland(东芬兰大学);Inria(法国国家信息与自动化研究所);University of Edinburgh(爱丁堡大学);NEC Corp.(日本电气股份有限公司)
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心)
Targeted Augmented Data for Audio Deepfake Detection
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学跨学科安全、可靠性与信任中心(SnT));Cristal Laboratory, National School of Computer Sciences (ENSI), Manouba University(马努巴大学计算机科学国家学院(ENSI)克里斯塔尔实验室)
Representation Loss Minimization with Randomized Selection Strategy for Efficient Environmental Fake Audio Detection
IIIT-Delhi(德里印度信息技术学院);UPES(Uttarakhand专业与技术大学);Reliance Jio AICoE(信实Jio人工智能卓越中心);University of Tartu(塔尔图大学);IIT-Dharwad(达尔瓦德印度理工学院);IIIT-Dharwad(达尔瓦德印度信息技术学院)
Low-rank Adaptation Method for Wav2vec2-based Fake Audio Detection
University of Science and Technology of China(中国科学技术大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Jiaotong University(北京交通大学);Tsinghua University(清华大学)
An Unsupervised Domain Adaptation Method for Locating Manipulated Region in partially fake Audio
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Automation, Tsinghua University(清华大学自动化系)
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, Chinese Academy of Sciences(中国科学院人工智能学院);School of Information and Communication Engineering, Communication University of China(中国传媒大学信息与通信工程学院);Beijing National Research Center for Information Science and Technology, Tsinghua University(北京信息科学与技术国家研究中心,清华大学);Department of Automation, Tsinghua University(清华大学自动化系);School of New Media and Communication, Tianjin University(天津大学新媒体与传播学院);Anhui Province Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University(安徽省多模态认知计算重点实验室,安徽大学计算机科学与技术学院);Tencent, AI Lab(腾讯,AI实验室)
ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Department of Automation, Tsinghua University(清华大学自动化系);Taizhou University(台州大学);Anhui University(安徽大学);Beijing National Research Center for Information Science and Technology,Tsinghua University(北京信息科学与技术国家研究中心,清华大学)
Fake Speech Wild: Detecting Deepfake Speech on Social Media Platform
Communication University of China(通信大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Institute of Technology(北京理工大学);Beijing National Research Center for Information Science and Technology(北京国家信息科学与技术研究中心);Beijing University of Posts and Telecommunications(北京邮电大学)
Deepfake Detection System for the ADD Challenge Track 3.2 Based on Score Fusion
Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);University of Chinese Academy of Sciences(中国科学院大学);Duke Kunshan University(昆山杜克大学);Wuhan University(武汉大学)
Deep Spectro-temporal Artifacts for Detecting Synthesized Speech
Tianjin University(天津大学);National Institute of Informatics(国立情报学研究所);Taiyuan University of Technology(太原理工大学);Japan Advanced Institute of Science and Technology(北陆先端科学技术大学院大学)