The recent emergence of deepfakes has brought manipulated and generated content to the forefront of machine learning research. Automatic detection of deepfakes has seen many new machine learning techniques, however, human detection capabilities are far less explored. In this paper, we present results from comparing the abilities of humans and machines for detecting audio deepfakes used to imitate someone's voice. For this, we use a web-based application framework formulated as a game. Participants were asked to distinguish between real and fake audio samples. In our experiment, 472 unique users competed against a state-of-the-art AI deepfake detection algorithm for 14912 total of rounds of the game. We find that humans and deepfake detection algorithms share similar strengths and weaknesses, both struggling to detect certain types of attacks. This is in contrast to the superhuman performance of AI in many application areas such as object detection or face recognition. Concerning human success factors, we find that IT professionals have no advantage over non-professionals but native speakers have an advantage over non-native speakers. Additionally, we find that older participants tend to be more susceptible than younger ones. These insights may be helpful when designing future cybersecurity training for humans as well as developing better detection algorithms.
Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy
Communication University of China(中国传媒大学);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学)
Audio Deepfake Attribution: An Initial Dataset and Investigation
School of Electronics and Information Engineering, Beihang University(北京航空航天大学电子信息工程学院);Tsinghua University(清华大学);National Research Center for Information Science and Technology, Tsinghua University(清华大学国家信息科学与技术研究中心)
AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan
Communication University of China(中国传媒大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Institute of Technology(北京理工大学);Shanghai Jiao Tong University(上海交通大学)
Targeted Augmented Data for Audio Deepfake Detection
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学跨学科安全、可靠性与信任中心(SnT));Cristal Laboratory, National School of Computer Sciences (ENSI), Manouba University(马努巴大学计算机科学国家学院(ENSI)克里斯塔尔实验室)
ADD 2023: the Second Audio Deepfake Detection Challenge
Peoples' Friendship University of Russia (RUDN University)(俄罗斯人民友谊大学(RUDN大学));Joint Institute for Nuclear Research(联合核研究所);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Skövde(舍夫德大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong(香港中文大学)
Fraunhofer AISEC(弗劳恩霍夫人工智能安全研究所);Wrocław University of Science and Technology(沃斯克拉大学科学与技术学院);Resemble AI(Resemble AI公司);Technical University of Cluj-Napoca(克鲁克斯-纳波卡技术大学);Neodyme AG(Neodyme公司);TU Munich(慕尼黑技术大学)
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Beijing Jiaotong University(北京交通大学);Hebei University of Technology(河北工业大学)
Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection
Indian Institute of Technology, Ropar, India(印度理工学院罗帕尔分校);Machine Intelligence Group, Birla Institute of Technology and Science, Pilani, Hyderabad Campus, India(比拉理工科学院帕利尼 Hyderabad 分校机器智能小组);Monash University, Melbourne, Australia(墨尔本大学)
What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Computer and Information Technology, University of Beijing Jiaotong(北京交通大学计算机与信息技术学院);Department of Automation, Tsinghua University(清华大学自动化系);University of Science and Technology of China(中国科学技术大学)
Gender Fairness in Audio Deepfake Detection: Performance and Disparity Analysis
School of Computing, Wichita State University(维斯科大学计算学院);Institut national de la recherche scientifique (INRS–EMT)(国家科学研究中心(INRS–EMT));INRS-UQO Mixed Research Unit on Cybersecurity(网络安全混合研究单位(INRS-UQO))