Current text-to-speech algorithms produce realistic fakes of human voices, making deepfake detection a much-needed area of research. While researchers have presented various techniques for detecting audio spoofs, it is often unclear exactly why these architectures are successful: Preprocessing steps, hyperparameter settings, and the degree of fine-tuning are not consistent across related work. Which factors contribute to success, and which are accidental? In this work, we address this problem: We systematize audio spoofing detection by re-implementing and uniformly evaluating architectures from related work. We identify overarching features for successful audio deepfake detection, such as using cqtspec or logspec features instead of melspec features, which improves performance by 37% EER on average, all other factors constant. Additionally, we evaluate generalization capabilities: We collect and publish a new dataset consisting of 37.9 hours of found audio recordings of celebrities and politicians, of which 17.2 hours are deepfakes. We find that related work performs poorly on such real-world data (performance degradation of up to one thousand percent). This may suggest that the community has tailored its solutions too closely to the prevailing ASVSpoof benchmark and that deepfakes are much harder to detect outside the lab than previously thought.
Targeted Augmented Data for Audio Deepfake Detection
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg（卢森堡大学跨学科安全、可靠性与信任中心（SnT））；Cristal Laboratory, National School of Computer Sciences (ENSI), Manouba University（马努巴大学计算机科学国家学院（ENSI）克里斯塔尔实验室）
Wrocław University of Science and Technology（弗罗茨瓦夫科技大学）
Audio DeepFakes are utterances generated with the use of deep neural networks. They are highly misleading and pose a threat due to use in fake news, impersonation, or extortion. In this work, we focus on increasing accessibility to the audio DeepFake detection methods by providing SpecRNet, a neural network architecture characterized by a quick inference time and low computational requirements. Our benchmark shows that SpecRNet, requiring up to about 40% less time to process an audio sample, provides performance comparable to LCNN architecture - one of the best audio DeepFake detection models.…
Deepfake is content or material that is generated or manipulated using AI methods, to pass off as real. There are four different deepfake types: audio, video, image and text. In this research we focus on audio deepfakes and how people perceive it. There are several audio deepfake generation frameworks, but we chose MelGAN which is a non-autoregressive and fast audio deepfake generating framework, requiring fewer parameters. This study tries to assess audio deepfake perceptions among college students from different majors. This study also answers the question of how their background and major c…
University of Maryland Baltimore County（马里兰大学巴尔的摩郡分校）
Deepfake is content or material that is synthetically generated or manipulated using artificial intelligence (AI) methods, to be passed off as real and can include audio, video, image, and text synthesis. This survey has been conducted with a different perspective compared to existing survey papers, that mostly focus on just video and image deepfakes. This survey not only evaluates generation and detection methods in the different deepfake categories, but mainly focuses on audio deepfakes that are overlooked in most of the existing surveys. This paper critically analyzes and provides a unique …
This paper describes our submitted systems to the 2022 ADD challenge withing the tracks 1 and 2. Our approach is based on the combination of a pre-trained wav2vec2 feature extractor and a downstream classifier to detect spoofed audio. This method exploits the contextualized speech representations at the different transformer layers to fully capture discriminative information. Furthermore, the classification model is adapted to the application scenario using different data augmentation techniques. We evaluate our system for audio synthesis detection in both the ASVspoof 2021 and the 2022 ADD ch…
What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Computer and Information Technology, University of Beijing Jiaotong（北京交通大学计算机与信息技术学院）；Department of Automation, Tsinghua University（清华大学自动化系）；University of Science and Technology of China（中国科学技术大学）
ADD 2023: the Second Audio Deepfake Detection Challenge
Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（舍夫德大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tsinghua University（清华大学）；National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）
Anhui University（安徽大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所）；Qiyuan Laboratory（启元实验室）
Recently, pioneer research works have proposed a large number of acoustic features (log power spectrogram, linear frequency cepstral coefficients, constant Q cepstral coefficients, etc.) for audio deepfake detection, obtaining good performance, and showing that different subbands have different contributions to audio deepfake detection. However, this lacks an explanation of the specific information in the subband, and these features also lose information such as phase. Inspired by the mechanism of synthetic speech, the fundamental frequency (F0) information is used to improve the quality of sy…
Audio DeepFakes allow the creation of high-quality, convincing utterances and therefore pose a threat due to its potential applications such as impersonation or fake news. Methods for detecting these manipulations should be characterized by good generalization and stability leading to robustness against attacks conducted with techniques that are not explicitly included in the training. In this work, we introduce Attack Agnostic Dataset - a combination of two audio DeepFakes and one anti-spoofing datasets that, thanks to the disjoint use of attacks, can lead to better generalization of detectio…
Deep generative modeling has the potential to cause significant harm to society. Recognizing this threat, a magnitude of research into detecting so-called "Deepfakes" has emerged. This research most often focuses on the image domain, while studies exploring generated audio signals have, so-far, been neglected. In this paper we make three key contributions to narrow this gap. First, we provide researchers with an introduction to common signal processing techniques used for analyzing audio signals. Second, we present a novel data set, for which we collected nine sample sets from five different n…
Significant advancements made in the generation of deepfakes have caused security and privacy issues. Attackers can easily impersonate a person's identity in an image by replacing his face with the target person's face. Moreover, a new domain of cloning human voices using deep-learning technologies is also emerging. Now, an attacker can generate realistic cloned voices of humans using only a few seconds of audio of the target person. With the emerging threat of potential harm deepfakes can cause, researchers have proposed deepfake detection methods. However, they only focus on detecting a sing…
Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy
Communication University of China（中国传媒大学）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）
Audio Deepfake Attribution: An Initial Dataset and Investigation
School of Electronics and Information Engineering, Beihang University（北京航空航天大学电子信息工程学院）；Tsinghua University（清华大学）；National Research Center for Information Science and Technology, Tsinghua University（清华大学国家信息科学与技术研究中心）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Beijing Jiaotong University（北京交通大学）；Hebei University of Technology（河北工业大学）
Fraunhofer AISEC（弗劳恩霍夫应用安全研究中心）；Technical University of Munich（慕尼黑工业大学）；University of Southampton（南安普顿大学）
Deepfakes are synthetically generated media often devised with malicious intent. They have become increasingly more convincing with large training datasets advanced neural networks. These fakes are readily being misused for slander, misinformation and fraud. For this reason, intensive research for developing countermeasures is also expanding. However, recent work is almost exclusively limited to deepfake detection - predicting if audio is real or fake. This is despite the fact that attribution (who created which fake?) is an essential building block of a larger defense strategy, as practiced i…
Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection
Indian Institute of Technology, Ropar, India（印度理工学院罗帕尔分校）；Machine Intelligence Group, Birla Institute of Technology and Science, Pilani, Hyderabad Campus, India（比拉理工科学院帕利尼 Hyderabad 分校机器智能小组）；Monash University, Melbourne, Australia（墨尔本大学）
RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tsinghua University（清华大学）
Boundary and Intra-Segment Learning for Partial Audio Deepfake Localization
Sun Yat-sen University（中山大学）；China Mobile Internet Corporation（中国移动互联网公司）；The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）
AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan
Communication University of China（中国传媒大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Institute of Technology（北京理工大学）；Shanghai Jiao Tong University（上海交通大学）
