With the rapid advancement of technologies like text-to-speech (TTS) and voice conversion (VC), detecting deepfake voices has become increasingly crucial. However, both academia and industry lack a comprehensive and intuitive benchmark for evaluating detectors. Existing datasets are limited in language diversity and lack many manipulations encountered in real-world production environments.
To fill this gap, we propose VoiceWukong, a benchmark designed to evaluate the performance of deepfake voice detectors. To build the dataset, we first collected deepfake voices generated by 19 advanced and widely recognized commercial tools and 15 open-source tools. We then created 38 data variants covering six types of manipulations, constructing the evaluation dataset for deepfake voice detection. VoiceWukong thus includes 265,200 English and 148,200 Chinese deepfake voice samples. Using VoiceWukong, we evaluated 12 state-of-the-art detectors. AASIST2 achieved the best equal error rate (EER) of 13.50%, while all others exceeded 20%. Our findings reveal that these detectors face significant challenges in real-world applications, with dramatically declining performance. In addition, we conducted a user study with more than 300 participants. The results are compared with the performance of the 12 detectors and a multimodel large language model (MLLM), i.e., Qwen2-Audio, where different detectors and humans exhibit varying identification capabilities for deepfake voices at different deception levels, while the LALM demonstrates no detection ability at all. Furthermore, we provide a leaderboard for deepfake voice detection, publicly available at {https://voicewukong.github.io}.
Enkidu: Universal Frequential Perturbation for Real-Time Audio Privacy Protection against Voice Deepfakes
College of Computer Science；Technology, Zhejiang University Hangzhou China；School of Big Data \& Software Engineering, Chongqing University Chongqing China；Technology, Zhejiang University；School of Big Data \& Software Engineering, Chongqing University
Mitigating Unauthorized Speech Synthesis for Voice Protection
Beijing University of Posts and Telecommunications（北京邮电大学）；CSIRO’s Data61（联邦科学与工业研究组织Data61）；National University of Singapore（新加坡国立大学）；The University of Hong Kong（香港大学）
Hindi audio-video-Deepfake (HAV-DF): A Hindi language-based Audio-video Deepfake Dataset
BML Munjal University（BML蒙贾尔大学）；School of Engineering & Technology, BML Munjal University（BML蒙贾尔大学工程与技术学院）；Center for Advanced Data and Computational Science, BML Munjal University（BML蒙贾尔大学高级数据与计算科学中心）
SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis
Beijing University of Posts and Telecommunications（北京邮电大学）；CSIRO’s Data61（联邦科学与工业研究组织数据61）；National University of Singapore（新加坡国立大学）；The University of Hong Kong（香港大学）
Zero-Shot to Zero-Lies: Detecting Bengali Deepfake Audio through Transfer Learning
Department of CSE, BRAC University, Bangladesh（布鲁斯大学计算机科学与工程系）；Department of CSE, Ahsanullah University of Science and Technology, Bangladesh（阿沙努拉科技大学计算机科学与工程系）；Enosis Solutions, Bangladesh（恩奥斯解决方案公司）；Department of CSE, Southeast University, Bangladesh（东南大学计算机科学与工程系）
D-CAPTCHA++: A Study of Resilience of Deepfake CAPTCHA under Transferable Imperceptible Adversarial Attack
School of Computer Science（计算机学院）；University College Dublin（都柏林大学学院）；School of Computer Science and Statistics（计算机与统计学院）；Trinity College Dublin（都柏林圣三一学院）；University of Science（科学大学）
Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；East China Normal University（华东师范大学）；Kyushu University（九州大学）；Institute of Computing Innovation, Zhejiang University（浙江大学计算创新研究院）
With the recent advances in voice synthesis, AI-synthesized fake voices are indistinguishable to human ears and widely are applied to produce realistic and natural DeepFakes, exhibiting real threats to our society. However, effective and robust detectors for synthesized fake voices are still in their infancy and are not ready to fully tackle this emerging threat. In this paper, we devise a novel approach, named \emph{DeepSonar}, based on monitoring neuron behaviors of speaker recognition (SR) system, \ie, a deep neural network (DNN), to discern AI-synthesized fake voices. Layer-wise neuron beh…
ESDD2: Environment-Aware Speech and Sound Deepfake Detection Challenge Evaluation Plan
Duke Kunshan University（杜克-肯斯敦大学）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；The University of Melbourne（墨尔本大学）；Johns Hopkins University（约翰霍普金斯大学）；Fortemedia Singapore（新加坡Fortemedia）
Offenburg University of Applied Sciences（奥芬堡应用技术大学）
Synthesizing voice with the help of machine learning techniques has made rapid progress over the last years [1] and first high profile fraud cases have been recently reported [2]. Given the current increase in using conferencing tools for online teaching, we question just how easy (i.e. needed data, hardware, skill set) it would be to create a convincing voice fake. We analyse how much training data a participant (e.g. a student) would actually need to fake another participants voice (e.g. a professor). We provide an analysis of the existing state of the art in creating voice deep fakes, as we…
I Can Hear You: Selective Robust Training for Deepfake Audio Detection
Columbia University（哥伦比亚大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；New York University（纽约大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Rutgers University（罗格斯大学）
Textual Acoustic Grounding for Generalizable LLM-Based Deepfake Voice Detection
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；National Institute of Informatics（情报信息学研究所）；Technical University of Berlin（柏林工业大学）
Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis
School of Computing, Wichita State University（维斯科大学计算学院）；Institut national de la recherche scientifique (INRS–EMT)（国家科学研究中心（INRS–EMT））；INRS-UQO Mixed Research Unit on Cybersecurity（网络安全混合研究单位（INRS-UQO））
Profiling the Voice: Speaker-Specific Phoneme Fingerprinting for Speech Deepfake Detection
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education（航空航天信息安全与可信计算重点实验室，教育部）；School of Cyber Science and Engineering, Wuhan University（武汉大学计算机科学与工程学院）
While the significant advancements have made in the generation of deepfakes using deep learning technologies, its misuse is a well-known issue now. Deepfakes can cause severe security and privacy issues as they can be used to impersonate a person's identity in a video by replacing his/her face with another person's face. Recently, a new problem of generating synthesized human voice of a person is emerging, where AI-based deep learning models can synthesize any person's voice requiring just a few seconds of audio. With the emerging threat of impersonation attacks using deepfake audios and video…
