This paper reports the first successful application of a differentiable architecture search (DARTS) approach to the deepfake and spoofing detection problems. An example of neural architecture search, DARTS operates upon a continuous, differentiable search space which enables both the architecture and parameters to be optimised via gradient descent. Solutions based on partially-connected DARTS use random channel masking in the search space to reduce GPU time and automatically learn and optimise complex neural architectures composed of convolutional operations and residual blocks. Despite being learned quickly with little human effort, the resulting networks are competitive with the best performing systems reported in the literature. Some are also far less complex, containing 85% fewer parameters than a Res2Net competitor.
Despite several years of research in deepfake and spoofing detection for automatic speaker verification, little is known about the artefacts that classifiers use to distinguish between bona fide and spoofed utterances. An understanding of these is crucial to the design of trustworthy, explainable solutions. In this paper we report an extension of our previous work to better understand classifier behaviour to the use of SHapley Additive exPlanations (SHAP) to attack analysis. Our goal is to identify the artefacts that characterise utterances generated by different attacks algorithms. Using a pa…
Artefacts that serve to distinguish bona fide speech from spoofed or deepfake speech are known to reside in specific subbands and temporal segments. Various approaches can be used to capture and model such artefacts, however, none works well across a spectrum of diverse spoofing attacks. Reliable detection then often depends upon the fusion of multiple detection systems, each tuned to detect different forms of attack. In this paper we show that better performance can be achieved when the fusion is performed within the model itself and when the representation is learned automatically from raw w…
Towards Scalable AASIST: Refining Graph Attention for Speech Deepfake Detection
Speechka.ai Information Technologies, Mechanics and Optics University（Speechka.ai 信息科技，力学与光学大学）；St. Petersburg State University（圣彼得堡州立大学）；Moscow Technical University of Communications and Informatics（莫斯科通信与信息学技术大学）
EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection
School of Cyber Science and Engineering, Wuhan University（武汉大学网络安全科学与工程学院）；Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education（教育部航空航天信息安全与可信计算重点实验室）
Audio DeepFakes allow the creation of high-quality, convincing utterances and therefore pose a threat due to its potential applications such as impersonation or fake news. Methods for detecting these manipulations should be characterized by good generalization and stability leading to robustness against attacks conducted with techniques that are not explicitly included in the training. In this work, we introduce Attack Agnostic Dataset - a combination of two audio DeepFakes and one anti-spoofing datasets that, thanks to the disjoint use of attacks, can lead to better generalization of detectio…
ASVspoof 2021 is the forth edition in the series of bi-annual challenges which aim to promote the study of spoofing and the design of countermeasures to protect automatic speaker verification systems from manipulation. In addition to a continued focus upon logical and physical access tasks in which there are a number of advances compared to previous editions, ASVspoof 2021 introduces a new task involving deepfake speech detection. This paper describes all three tasks, the new databases for each of them, the evaluation metrics, four challenge baselines, the evaluation platform and a summary of …
Profiling the Voice: Speaker-Specific Phoneme Fingerprinting for Speech Deepfake Detection
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education（航空航天信息安全与可信计算重点实验室，教育部）；School of Cyber Science and Engineering, Wuhan University（武汉大学计算机科学与工程学院）
Duke University（杜克大学）；Duke Kunshan University（昆山杜克大学）
The present paper proposes a waveform boundary detection system for audio spoofing attacks containing partially manipulated segments. Partially spoofed/fake audio, where part of the utterance is replaced, either with synthetic or natural audio clips, has recently been reported as one scenario of audio deepfakes. As deepfakes can be a threat to social security, the detection of such spoofing audio is essential. Accordingly, we propose to address the problem with a deep learning-based frame-level detection system that can detect partially spoofed audio and locate the manipulated pieces. Our prop…
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild
Carnegie Mellon University（卡内基梅隆大学）；Renmin University of China（中国人民大学）；National Institute of Informatics（信息学研究所）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Meta；University of Tokyo（东京大学）；EURECOM（欧洲通信学院）；Shanghai Jiao Tong University（上海交通大学）；Yonsei University（延世大学）；Keio University（庆应义塾大学）
Self-supervised speech model is a rapid progressing research topic, and many pre-trained models have been released and used in various down stream tasks. For speech anti-spoofing, most countermeasures (CMs) use signal processing algorithms to extract acoustic features for classification. In this study, we use pre-trained self-supervised speech models as the front end of spoofing CMs. We investigated different back end architectures to be combined with the self-supervised front end, the effectiveness of fine-tuning the front end, and the performance of using different pre-trained self-supervise…
State-of-the-art methods for audio generation suffer from fingerprint artifacts and repeated inconsistencies across temporal and spectral domains. Such artifacts could be well captured by the frequency domain analysis over the spectrogram. Thus, we propose a novel use of long-range spectro-temporal modulation feature -- 2D DCT over log-Mel spectrogram for the audio deepfake detection. We show that this feature works better than log-Mel spectrogram, CQCC, MFCC, as a suitable candidate to capture such artifacts. We employ spectrum augmentation and feature normalization to decrease overfitting an…
Generalizable Speech Deepfake Detection via Information Bottleneck Enhanced Adversarial Alignment
School of Information and Electronic Engineering, Zhejiang Gongshang University, Hangzhou, China（信息与电子工程学院，浙江工商大学，杭州，中国）；Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, USA（电气与计算机工程系，新泽西理工学院，纽卡斯尔，美国）
Institute for Advancing Intelligence, TCG Centres for Research and Education in Science and Technology（TCG科学与技术研究教育中心智能推进研究所）；Indian Statistical Institute, Kolkata（印度统计研究所）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究院）
AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan
Communication University of China（中国传媒大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing Institute of Technology（北京理工大学）；Shanghai Jiao Tong University（上海交通大学）
Wav2DF-TSL: Two-stage Learning with Efficient Pre-training and Hierarchical Experts Fusion for Robust Audio Deepfake Detection
School of Computer Science and Technology, Xinjiang University, Urumqi, China（新疆大学计算机科学与技术学院）；Hefei iFly Digital Technology Co. Ltd., Hefei, China（合肥iFly数字技术有限公司）；University of Science and Technology of China, Hefei, China（中国科学技术大学）；Department of Electronic Engineering, Tsinghua University, Beijing, China（清华大学电子工程系）
ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild
Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；LORIA（洛林计算机科学及其应用实验室）；National Institute of Informatics（国立信息学研究所）；Institute for Advancing Intelligence, TCG CREST（TCG CREST 智能推进研究所）；EURECOM（欧洲通信学院）；Cerence Inc.（赛轮思公司）；Avignon Université（阿维尼翁大学）；Nuance Communications（Nuance 通信公司）；University of Eastern Finland（东芬兰大学）
Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detection
Nanyang Technological University, Singapore（南洋理工大学，新加坡）；National University of Singapore, Singapore（国立新加坡大学，新加坡）；The Hong Kong Polytechnic University, Hong Kong（香港理工大学，香港）
Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing
Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电子与计算机工程系）；LIGHTSPEED, Singapore（新加坡LIGHTSPEED公司）；Nanyang Technological University, Singapore（南洋理工大学）；Fortemedia Singapore, Singapore（新加坡Fortemedia公司）；Department of Electrical and Electronic Engineering and the Research Centre for Data Science & Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学电子与电气工程系及数据科学与人工智能研究中心）；Shenzhen Research Institute of Big Data, School of Artificial Intelligence, School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）大数据研究院、人工智能学院、数据科学学院）
