This technical report details changes applied to a noise filter to facilitate its application and improve its results. The filter is applied to denoise natural sounds recorded in the wild and to generate an acoustic index used in soundscape analysis.
2-Dimensional spectral gating for denoising bioacoustics recordings
Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Pioneer Centre for AI, University of Copenhagen(哥本哈根大学人工智能先锋中心);McGill University(麦吉尔大学);Stockholm University(斯德哥尔摩大学)
Enhancing the sound quality of historical music recordings is a long-standing problem. This paper presents a novel denoising method based on a fully-convolutional deep neural network. A two-stage U-Net model architecture is designed to model and suppress the degradations with high fidelity. The method processes the time-frequency representation of audio, and is trained using realistic noisy data to jointly remove hiss, clicks, thumps, and other common additive disturbances from old analog discs. The proposed model outperforms previous methods in both objective and subjective metrics. The resul…
School of Mathematics and Statistics, Victoria University of Wellington(惠灵顿维多利亚大学数学与统计学院)
Sound recordings are used in various ecological studies, including acoustic wildlife monitoring. Such surveys require automatic detection of target sound events. However, current detectors, especially those relying on band-limited energy, are severely impacted by wind. The rapid dynamics of this noise invalidate standard noise estimators, and no satisfactory method for dealing with it exists in bioacoustics, where simple training and generalization between conditions are important. We propose to estimate the transient noise level by fitting short-term spectrum models to a wavelet packet repres…
Department of Electrical Engineering, Columbia University(哥伦比亚大学电气工程系);Google Research(谷歌研究院)
This work describes a speech denoising system for machine ears that aims to improve speech intelligibility and the overall listening experience in noisy environments. We recorded approximately 100 hours of audio data with reverberation and moderate environmental noise using a pair of microphone arrays placed around each of the two ears and then mixed sound recordings to simulate adverse acoustic scenes. Then, we trained a multi-channel speech denoising network (MCSDN) on the mixture of recordings. To improve the training, we employ an unsupervised method, complex angular central Gaussian mixtu…
Politecnico di Milano(米兰理工大学);Fraunhofer Institute for Digital Media Technology(弗劳恩霍夫数字媒体技术研究所);TU Ilmenau(伊尔梅瑙工业大学)
This work proposes a method for source device identification from speech recordings that applies neural-network-based denoising, to mitigate the impact of counter-forensics attacks using noise injection. The method is evaluated by comparing the impact of denoising on three state-of-the-art features for microphone classification, determining their discriminating power with and without denoising being applied. The proposed framework achieves a significant performance increase for noisy material, and more generally, validates the usefulness of applying denoising prior to device identification for…
Multi-Representation Attention Framework for Underwater Bioacoustic Denoising and Recognition
MILA – Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Université du Québec à Rimouski(里莫斯基大学);ISMER(ISMER机构);Institut Polytechnique de Paris(巴黎高等理工学院)
This paper tackles the problem of the heavy dependence of clean speech data required by deep learning based audio-denoising methods by showing that it is possible to train deep speech denoising networks using only noisy speech samples. Conventional wisdom dictates that in order to achieve good speech denoising performance, there is a requirement for a large quantity of both noisy speech samples and perfectly clean speech samples, resulting in a need for expensive audio recording equipment and extremely controlled soundproof recording studios. These requirements pose significant challenges in d…
Music discovery services let users identify songs from short mobile recordings. These solutions are often based on Audio Fingerprinting, and rely more specifically on the extraction of spectral peaks in order to be robust to a number of distortions. Few works have been done to study the robustness of these algorithms to background noise captured in real environments. In particular, AFP systems still struggle when the signal to noise ratio is low, i.e when the background noise is strong. In this project, we tackle this problematic with Deep Learning. We test a new hybrid strategy which consists…
Southeast University(东南大学);Centre de Recherche en Information Biomédicale Sino-français (CRIBs)(中法生物医学信息研究中心);The 28th Research Institute of China Electronics Technology Group Corporation(中国电子科技集团公司第二十八研究所);Univ Rennes(雷恩大学)
In traditional speech denoising tasks, clean audio signals are often used as the training target, but absolutely clean signals are collected from expensive recording equipment or in studios with the strict environments. To overcome this drawback, we propose an end-to-end self-supervised speech denoising training scheme using only noisy audio signals, named Only-Noisy Training (ONT), without extra training conditions. The proposed ONT strategy constructs training pairs only from each single noisy audio, and it contains two modules: training audio pairs generated module and speech denoising modu…
We propose a method for noise reduction, the task of producing a clean audio signal from a recording corrupted by additive noise. Many common approaches to this problem are based upon applying non-negative matrix factorization to spectrogram measurements. These methods use a noiseless recording, which is believed to be similar in structure to the signal of interest, and a pure-noise recording to learn dictionaries for the true signal and the noise. One may then construct an approximation of the true signal by projecting the corrupted recording on to the clean dictionary. In this work, we build…
We present a framework for detecting blue whale vocalisations from acoustic submarine recordings. The proposed methodology comprises three stages: i) a preprocessing step where the audio recordings are conditioned through normalisation, filtering, and denoising; ii) a label-propagation mechanism to ensure the consistency of the annotations of the whale vocalisations, and iii) a convolutional neural network that receives audio samples. Based on 34 real-world submarine recordings (28 for training and 6 for testing) we obtained promising performance indicators including an Accuracy of 85.4\% and …
Audio bandwidth extension aims to expand the spectrum of narrow-band audio signals. Although this topic has been broadly studied during recent years, the particular problem of extending the bandwidth of historical music recordings remains an open challenge. This paper proposes BEHM-GAN, a model based on generative adversarial networks, as a practical solution to this problem. The proposed method works with the complex spectrogram representation of audio and, thanks to a dedicated regularization strategy, can effectively extend the bandwidth of out-of-distribution real historical recordings. Th…
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model
University of Galway(高威大学);University of Wrocław(弗罗茨瓦夫大学);Wrocław University of Science and Technology(弗罗茨瓦夫理工大学);Tooploox(Tooploox公司);Trinity College Dublin(都柏林圣三一学院)
Enhancing Spectrogram Realism in Singing Voice Synthesis via Explicit Bandwidth Extension Prior to Vocoder
Institute for Clarity in Documentation(清晰文档研究所);Inria Paris-Rocquencourt(巴黎-罗克Quantin研究所);Rajiv Gandhi University(拉朱·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
DFingerNet: Noise-Adaptive Speech Enhancement for Hearing Aids
University of Augsburg(奥格斯堡大学);Technical University of Munich(慕尼黑工业大学);WS Audiology;Imperial College London(帝国理工学院);Munich Data Science Institute(慕尼黑数据科学研究所)
University of Augsburg(奥格斯堡大学);Imperial College London(帝国理工学院)
N-HANS is a Python toolkit for in-the-wild audio enhancement, including speech, music, and general audio denoising, separation, and selective noise or source suppression. The functionalities are realised based on two neural network models sharing the same architecture, but trained separately. The models are comprised of stacks of residual blocks, each conditioned on additional speech or environmental noise recordings for adapting to different unseen speakers or environments in real life. In addition to a Python API, a command line interface is provided to researchers and developers, both of wh…