Animal vocalization denoising is a task similar to human speech enhancement, which is relatively well-studied. In contrast to the latter, it comprises a higher diversity of sound production mechanisms and recording environments, and this higher diversity is a challenge for existing models. Adding to the challenge and in contrast to speech, we lack large and diverse datasets comprising clean vocalizations. As a solution we use as training data pseudo-clean targets, i.e. pre-denoised vocalizations, and segments of background noise without a vocalization. We propose a train set derived from bioacoustics datasets and repositories representing diverse species, acoustic environments, geographic regions. Additionally, we introduce a non-overlapping benchmark set comprising clean vocalizations from different taxa and noise samples. We show that that denoising models (demucs, CleanUNet) trained on pseudo-clean targets obtained with speech enhancement models achieve competitive results on the benchmarking set. We publish data, code, libraries, and demos at https://earthspecies.github.io/biodenoising/.
BioSEN: A Bio-acoustic Signal Enhancement Network for Animal Vocalizations
Graduate School of Bioresource and Bioenvironmental Science, Kyushu University（九州大学生物资源与生物环境科学研究生院）；Faculty of Agriculture, Kyushu University（九州大学农学部）；School of Information and Physical Sciences, University of Newcastle（新castle大学信息与物理科学学院）；International Institute for Carbon-Neutral Energy Research, Kyushu University（九州大学国际碳中性能源研究所）
The lack of annotated training data in bioacoustics hinders the use of large-scale neural network models trained in a supervised way. In order to leverage a large amount of unannotated audio data, we propose AVES (Animal Vocalization Encoder based on Self-Supervision), a self-supervised, transformer-based audio representation model for encoding animal vocalizations. We pretrain AVES on a diverse set of unannotated audio datasets and fine-tune them for downstream bioacoustics tasks. Comprehensive experiments with a suite of classification and detection tasks have shown that AVES outperforms all…
Modeling real-world sound is a fundamental problem in the creative use of machine learning and many other fields, including human speech processing and bioacoustics. Transformer-based generative models and some prior work (e.g., DDSP) are known to produce realistic sound, although they have limited control and are hard to interpret. As an alternative, we aim to use modular synthesizers, i.e., compositional, parametric electronic musical instruments, for modeling non-music sounds. However, inferring synthesizer parameters given a target sound, i.e., the parameter inference task, is not trivial …
Behavior-Conditioned Animal Vocalization Generation: Event-Aware Modeling and Multidimensional Evaluation
College of Artificial Intelligence, China University of Petroleum（中国石油大学（北京）人工智能学院）；College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Institute of Acoustics, Chinese Academy of Sciences（中国科学院声学研究所）
Research Collaboratory for Advanced Intelligent Systems（高级智能系统研究合作中心）；Institute of Computer Science, University of the Philippines Los Baños（菲律宾大学洛斯巴诺斯分校计算机科学学院）
In this research endeavor, it was hypothesized that the sound produced by animals during their vocalizations can be used as identifiers of the animal breed or species even if they sound the same to unaided human ear. To test this hypothesis, three artificial neural networks (ANNs) were developed using bioacoustics properties as inputs for the respective automatic identification of 13 bird species, eight dog breeds, and 11 frog species. Recorded vocalizations of these animals were collected and processed using several known signal processing techniques to convert the respective sounds into comp…
Science & Technology of Music and Sound (STMS), IRCAM/CNRS/Sorbonne Université（音乐与声音科技实验室（STMS），法国声学/音乐协调研究所（IRCAM）/法国国家科学研究中心（CNRS）/索邦大学）；Alta Voce SAS（Alta Voce SAS 公司）；HES-So Valais（瑞士西部应用科学与艺术大学瓦莱州分校）；Lyon Neuroscience Center（里昂神经科学研究中心）
Vocal arousal, the non-linear acoustic features taken on by human and animal vocalizations when highly aroused, has an important communicative function because it signals aversive states such as fear, pain or distress. In this work, we present a computationally-efficient, real-time voice transformation algorithm, ANGUS, which uses amplitude modulation and time-domain filtering to simulate roughness, an important component of vocal arousal, in arbitrary voice recordings. In a series of 4 studies, we show that ANGUS allows parametric control over the spectral features of roughness like the prese…
2-Dimensional spectral gating for denoising bioacoustics recordings
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Pioneer Centre for AI, University of Copenhagen（哥本哈根大学人工智能先锋中心）；McGill University（麦吉尔大学）；Stockholm University（斯德哥尔摩大学）
