arXiv 1709.006112017-09cs.SD

A Recurrent Encoder-Decoder Approach with Skip-filtering Connections for Monaural Singing Voice Separation

Fraunhofer IDMT（弗劳恩霍夫数字媒体技术研究所）；Tampere University of Technology（坦佩雷理工大学）；Technical University of Ilmenau（伊尔梅瑙工业大学）

Stylianos Ioannis Mimilakis, Konstantinos Drossos, Tuomas Virtanen, Gerald Schuller

英文摘要：

The objective of deep learning methods based on encoder-decoder architectures for music source separation is to approximate either ideal time-frequency masks or spectral representations of the target music source(s). The spectral representations are then used to derive time-frequency masks. In this work we introduce a method to directly learn time-frequency masks from an observed mixture magnitude spectrum. We employ recurrent neural networks and train them using prior knowledge only for the magnitude spectrum of the target source. To assess the performance of the proposed method, we focus on the task of singing voice separation. The results from an objective evaluation show that our proposed method provides comparable results to deep learning based methods which operate over complicated signal representations. Compared to previous methods that approximate time-frequency masks, our method has increased performance of signal to distortion ratio by an average of 3.8 dB.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Monaural Singing Voice Separation with Skip-Filtering Connections and Recurrent Inference of Time-Frequency Mask

Fraunhofer IDMT（弗劳恩霍夫集成电路研究所）；Tampere University of Technology（坦佩雷理工大学）；Université de Montréal（蒙特利尔大学）；INRS-EMT（加拿大国家科学研究院能源、材料与电信研究所）

Singing voice separation based on deep learning relies on the usage of time-frequency masking. In many cases the masking process is not a learnable function or is not encapsulated into the deep learning optimization. Consequently, most of the existing methods rely on a post processing step using the generalized Wiener filtering. This work proposes a method that learns and optimizes (during training) a source-dependent mask and does not need the aforementioned post processing step. We introduce a recurrent inference algorithm, a sparse transformation step to improve the mask generation process,…

2018-02-13cs.SDeess.AS

HTMD-Net: A Hybrid Masking-Denoising Approach to Time-Domain Monaural Singing Voice Separation

School of ECE, NTUA（雅典国立技术大学电气与计算机工程学院）

The advent of deep learning has led to the prevalence of deep neural network architectures for monaural music source separation, with end-to-end approaches that operate directly on the waveform level increasingly receiving research attention. Among these approaches, transformation of the input mixture to a learned latent space, and multiplicative application of a soft mask to the latent mixture, achieves the best performance, but is prone to the introduction of artifacts to the source estimate. To alleviate this problem, in this paper we propose a hybrid time-domain approach, termed the HTMD-N…

2021-03-07cs.SDeess.AS

Semi-Supervised Monaural Singing Voice Separation With a Masking Network Trained on Synthetic Mixtures

Tel Aviv University（特拉维夫大学）；Facebook AI Research（Facebook人工智能研究院）

We study the problem of semi-supervised singing voice separation, in which the training data contains a set of samples of mixed music (singing and instrumental) and an unmatched set of instrumental music. Our solution employs a single mapping function g, which, applied to a mixed sample, recovers the underlying instrumental music, and, applied to an instrumental sample, returns the same sample. The network g is trained using purely instrumental samples, as well as on synthetic mixed samples that are created by mixing reconstructed singing voices with random instrumental samples. Our results in…

2019-05-06cs.LGcs.SDeess.AS

MaD TwinNet: Masker-Denoiser Architecture with Twin Networks for Monaural Sound Source Separation

Tampere University of Technology（坦佩雷理工大学）；Fraunhofer IDMT – Technical University of Ilmenau（弗劳恩霍夫数字媒体技术研究所-伊尔梅瑙工业大学）；MILA, Université de Montréal（蒙特利尔大学米拉研究所）

Monaural singing voice separation task focuses on the prediction of the singing voice from a single channel music mixture signal. Current state of the art (SOTA) results in monaural singing voice separation are obtained with deep learning based methods. In this work we present a novel deep learning based method that learns long-term temporal patterns and structures of a musical piece. We build upon the recently proposed Masker-Denoiser (MaD) architecture and we enhance it with the Twin Networks, a technique to regularize a recurrent generative network using a backward running copy of the netwo…

2018-02-01cs.SDeess.AS

Depthwise Separable Convolutions Versus Recurrent Neural Networks for Monaural Singing Voice Separation

Tampere University（坦佩雷大学）；Fraunhofer-IDMT（弗劳恩霍夫数字媒体技术研究所）

Recent approaches for music source separation are almost exclusively based on deep neural networks, mostly employing recurrent neural networks (RNNs). Although RNNs are in many cases superior than other types of deep neural networks for sequence processing, they are known to have specific difficulties in training and parallelization, especially for the typically long sequences encountered in music source separation. In this paper we present a use-case of replacing RNNs with depth-wise separable (DWS) convolutions, which are a lightweight and faster variant of the typical convolutions. We focus…

2020-07-06cs.LGcs.SDeess.AS

Evolving Multi-Resolution Pooling CNN for Monaural Singing Voice Separation

Tianjin Polytechnic University（天津工业大学）；Japan Advanced Institute of Science and Technology（日本先端科学技术大学院大学）；University of Surrey（萨里大学）

Monaural Singing Voice Separation (MSVS) is a challenging task and has been studied for decades. Deep neural networks (DNNs) are the current state-of-the-art methods for MSVS. However, the existing DNNs are often designed manually, which is time-consuming and error-prone. In addition, the network architectures are usually pre-defined, and not adapted to the training data. To address these issues, we introduce a Neural Architecture Search (NAS) method to the structure design of DNNs for MSVS. Specifically, we propose a new multi-resolution Convolutional Neural Network (CNN) framework for MSVS n…

2020-08-03cs.LGcs.SDeess.AS

更多相关论文收起更多论文

Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation

University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Clarifai（Clarifai公司）；Adobe Research（奥多比研究院）

Monaural source separation is important for many real world applications. It is challenging because, with only a single channel of information available, without any constraints, an infinite number of solutions are possible. In this paper, we explore joint optimization of masking functions and deep recurrent neural networks for monaural source separation tasks, including monaural speech separation, monaural singing voice separation, and speech denoising. The joint optimization of the deep recurrent neural networks with an extra masking layer enforces a reconstruction constraint. Moreover, we e…

2015-10-01cs.AIcs.LGcs.MM

Examining the Mapping Functions of Denoising Autoencoders in Singing Voice Separation

Fraunhofer-IDMT（弗劳恩霍夫数字媒体技术研究所）；Tampere University（坦佩雷大学）

The goal of this work is to investigate what singing voice separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks based on the denoising autoencoder (DAE) model that are conditioned on the mixture magnitude spectra. To approximate the mapping functions, we propose an algorithm inspired by the knowledge distillation, denoted the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA, we examine the mapping functions of three fund…

2019-10-20cs.LGcs.SDeess.AS

A Vocoder Based Method For Singing Voice Extraction

Universitat Pompeu Fabra（庞培法布拉大学）；Joint Research Centre, European Commission（欧盟委员会联合研究中心）

This paper presents a novel method for extracting the vocal track from a musical mixture. The musical mixture consists of a singing voice and a backing track which may comprise of various instruments. We use a convolutional network with skip and residual connections as well as dilated convolutions to estimate vocoder parameters, given the spectrogram of an input mixture. The estimated parameters are then used to synthesize the vocal track, without any interference from the backing track. We evaluate our system, through objective metrics pertinent to audio quality and interference from backgrou…

2019-04-23cs.SDeess.AS

Improving singing voice separation using Deep U-Net and Wave-U-Net with data augmentation

Sorbonne Université（索邦大学）；IRCAM（声学/音乐研究和协作学院）；CNRS（法国国家科学研究中心）；Télécom ParisTech（巴黎高等电信学院）；Université Paris-Saclay（巴黎萨克雷大学）

State-of-the-art singing voice separation is based on deep learning making use of CNN structures with skip connections (like U-net model, Wave-U-Net model, or MSDENSELSTM). A key to the success of these models is the availability of a large amount of training data. In the following study, we are interested in singing voice separation for mono signals and will investigate into comparing the U-Net and the Wave-U-Net that are structurally similar, but work on different input representations. First, we report a few results on variations of the U-Net model. Second, we will discuss the potential of …

2019-03-04cs.SDeess.AS

Harmonic-Percussive Source Separation with Deep Neural Networks and Phase Recovery

Tampere University of Technology（坦佩雷理工大学）；Fraunhofer IDMT（弗劳恩霍夫数字媒体技术研究所）

Harmonic/percussive source separation (HPSS) consists in separating the pitched instruments from the percussive parts in a music mixture. In this paper, we propose to apply the recently introduced Masker-Denoiser with twin networks (MaD TwinNet) system to this task. MaD TwinNet is a deep learning architecture that has reached state-of-the-art results in monaural singing voice separation. Herein, we propose to apply it to HPSS by using it to estimate the magnitude spectrogram of the percussive source. Then, we retrieve the complex-valued short-time Fourier transform of the sources by means of a…

2018-07-30cs.SDeess.AS

SkipConvGAN: Monaural Speech Dereverberation using Generative Adversarial Networks via Complex Time-Frequency Masking

University of Texas at Dallas（德克萨斯大学达拉斯分校）

With the advancements in deep learning approaches, the performance of speech enhancing systems in the presence of background noise have shown significant improvements. However, improving the system's robustness against reverberation is still a work in progress, as reverberation tends to cause loss of formant structure due to smearing effects in time and frequency. A wide range of deep learning-based systems either enhance the magnitude response and reuse the distorted phase or enhance complex spectrogram using a complex time-frequency mask. Though these approaches have demonstrated satisfactor…

2022-11-22cs.LGcs.SDeess.AS

MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation

Sony Corporation（索尼公司）；Sony India Software Centre（索尼印度软件中心）

Deep neural networks have become an indispensable technique for audio source separation (ASS). It was recently reported that a variant of CNN architecture called MMDenseNet was successfully employed to solve the ASS problem of estimating source amplitudes, and state-of-the-art results were obtained for DSD100 dataset. To further enhance MMDenseNet, here we propose a novel architecture that integrates long short-term memory (LSTM) in multiple scales with skip connections to efficiently model long-term structures within an audio context. The experimental results show that the proposed method out…

2018-05-29cs.SDeess.AS

基于复卷积块注意力模块和联合时频损失的单声道语音增强

Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses

Alibaba Group（阿里巴巴集团）

针对单声道语音增强任务，提出轻量级复卷积块注意力模块（CCBAM）并结合联合时频损失函数，集成到深度复U-Net和CRN中形成新端到端框架，实验验证其性能优越。

2024-11-28cs.LGcs.SDeess.AS

鲁棒一次性歌声转换

Robust One-Shot Singing Voice Conversion

针对失真歌声下未见过歌手的高质量歌声转换难题，提出基于生成对抗网络的鲁棒一次性SVC模型、两阶段Robustify训练法及分层扩散声码器，显著提升转换质量与抗失真鲁棒性。

2023-10-06cs.LGcs.SDeess.AS

NaturalSpeech 2：潜在扩散模型是自然且零样本的语音与歌声合成器

NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers

Microsoft Research Asia（微软亚洲研究院）；Microsoft Azure Speech（微软Azure语音部门）

NaturalSpeech 2利用残差向量量化神经音频编解码器与扩散模型，结合语音提示机制实现零样本语音与歌声合成，在44K小时数据上训练后显著优于以往TTS系统。

2023-05-30cs.AIcs.CLcs.LG