首页/论文/2012.01576

arXiv 2012.015762020-12-02cs.SDcs.LGeess.AS

Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks

The Graduate Center, City University of New York（纽约城市大学研究生中心）；Brooklyn College, City University of New York（纽约城市大学布鲁克林学院）

Felix Grezes, Zhaoheng Ni, Viet Anh Trinh, Michael Mandel

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Recent works have shown that Deep Recurrent Neural Networks using the LSTM architecture can achieve strong single-channel speech enhancement by estimating time-frequency masks. However, these models do not naturally generalize to multi-channel inputs from varying microphone configurations. In contrast, spatial clustering techniques can achieve such generalization but lack a strong signal model. Our work proposes a combination of the two approaches. By using LSTMs to enhance spatial clustering based time-frequency masks, we achieve both the signal modeling performance of multiple single-channel LSTM-DNN speech enhancers and the signal separation performance and generality of multi-channel spatial clustering. We compare our proposed system to several baselines on the CHiME-3 dataset. We evaluate the quality of the audio from each system using SDR from the BSS\_eval toolkit and PESQ. We evaluate the intelligibility of the output of each system using word error rate from a Kaldi automatic speech recognizer.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Combining Spatial Clustering with LSTM Speech Models for Multichannel Speech Enhancement

The Graduate Center, City University of New York（纽约城市大学研究生中心）；Brooklyn College, City University of New York（纽约城市大学布鲁克林学院）

Recurrent neural networks using the LSTM architecture can achieve significant single-channel noise reduction. It is not obvious, however, how to apply them to multi-channel inputs in a way that can generalize to new microphone configurations. In contrast, spatial clustering techniques can achieve such generalization, but lack a strong signal model. This paper combines the two approaches to attain both the spatial separation performance and generality of multichannel spatial clustering and the signal modeling performance of multiple parallel single-channel LSTM speech enhancers. The system is c…

2020-12-02cs.LGcs.SDeess.AS

Improved MVDR Beamforming Using LSTM Speech Models to Clean Spatial Clustering Masks

The Graduate Center, City University of New York（纽约市立大学研究生中心）；Brooklyn College, City University of New York（纽约市立大学布鲁克林学院）

Spatial clustering techniques can achieve significant multi-channel noise reduction across relatively arbitrary microphone configurations, but have difficulty incorporating a detailed speech/noise model. In contrast, LSTM neural networks have successfully been trained to recognize speech from noise on single-channel inputs, but have difficulty taking full advantage of the information in multi-channel recordings. This paper integrates these two approaches, training LSTM speech models to clean the masks generated by the Model-based EM Source Separation and Localization (MESSL) spatial clustering…

2020-12-02cs.LGcs.SDeess.AS

Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training

Microsoft（微软）

Recently, masked prediction pre-training has seen remarkable progress in self-supervised learning (SSL) for speech recognition. It usually requires a codebook obtained in an unsupervised way, making it less accurate and difficult to interpret. We propose two supervision-guided codebook generation approaches to improve automatic speech recognition (ASR) performance and also the pre-training efficiency, either through decoding with a hybrid ASR system to generate phoneme-level alignments (named PBERT), or performing clustering on the supervised speech features extracted from an end-to-end CTC mo…

2022-06-21cs.CLcs.SDeess.AS

LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization

Duke Kunshan University（昆山杜克大学）；Sun Yat-sen University（中山大学）；LIMSI（法国计算机科学与自动化研究所）；CNRS（法国国家科学研究中心）；Univ. Paris-Sud（巴黎第十一大学）

More and more neural network approaches have achieved considerable improvement upon submodules of speaker diarization system, including speaker change detection and segment-wise speaker embedding extraction. Still, in the clustering stage, traditional algorithms like probabilistic linear discriminant analysis (PLDA) are widely used for scoring the similarity between two speech segments. In this paper, we propose a supervised method to measure the similarity matrix between all segments of an audio recording with sequential bidirectional long short-term memory networks (Bi-LSTM). Spectral cluste…

2019-07-23cs.LGcs.SDeess.AS

Time-Frequency Mask Aware Bi-directional LSTM: A Deep Learning Approach for Underwater Acoustic Signal Separation

National Key Laboratory of Science and Technology on Communications（通信抗干扰技术国家级重点实验室）；University of Electronic Science and Technology of China（电子科技大学）

The underwater acoustic signals separation is a key technique for the underwater communications. The existing methods are mostly model-based, and could not accurately characterise the practical underwater acoustic communication environment. They are only suitable for binary signal separation, but cannot handle multivariate signal separation. On the other hand, the recurrent neural network (RNN) shows powerful capability in extracting the features of the temporal sequences. Inspired by this, in this paper, we present a data-driven approach for underwater acoustic signals separation using deep l…

2022-02-09cs.SDeess.ASeess.SP

Dilated U-net based approach for multichannel speech enhancement from First-Order Ambisonics recordings

Orange Labs（奥兰治实验室）

We present a CNN architecture for speech enhancement from multichannel first-order Ambisonics mixtures. The data-dependent spatial filters, deduced from a mask-based approach, are used to help an automatic speech recognition engine to face adverse conditions of reverberation and competitive speakers. The mask predictions are provided by a neural network, fed with rough estimations of speech and noise amplitude spectra, under the assumption of known directions of arrival. This study evaluates the replacing of the recurrent LSTM network previously investigated by a convolutive U-net under more s…

2020-06-02cs.SDeess.ASeess.SP

更多相关论文收起更多论文

DNN driven Speaker Independent Audio-Visual Mask Estimation for Speech Separation

University of Stirling（斯特林大学）；Aix Marseille Univ（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）；University of Sheffield（谢菲尔德大学）

Human auditory cortex excels at selectively suppressing background noise to focus on a target speaker. The process of selective attention in the brain is known to contextually exploit the available audio and visual cues to better focus on target speaker while filtering out other noises. In this study, we propose a novel deep neural network (DNN) based audiovisual (AV) mask estimation model. The proposed AV mask estimation model contextually integrates the temporal dynamics of both audio and noise-immune visual features for improved mask estimation and speech separation. For optimal AV features…

2018-07-31cs.CVcs.LGcs.SD

Low-Latency Deep Clustering For Speech Separation

Tampere University of Technology（坦佩雷理工大学）

This paper proposes a low algorithmic latency adaptation of the deep clustering approach to speaker-independent speech separation. It consists of three parts: a) the usage of long-short-term-memory (LSTM) networks instead of their bidirectional variant used in the original work, b) using a short synthesis window (here 8 ms) required for low-latency operation, and, c) using a buffer in the beginning of audio mixture to estimate cluster centres corresponding to constituent speakers which are then utilized to separate speakers within the rest of the signal. The buffer duration would serve as an i…

2019-02-19cs.SDeess.AS

Mask Detection and Breath Monitoring from Speech: on Data Augmentation, Feature Representation and Modeling

Duke Kunshan University（昆山杜克大学）；Wuhan University（武汉大学）；Lenovo Research（联想研究院）；Sun Yat-sen University（中山大学）

This paper introduces our approaches for the Mask and Breathing Sub-Challenge in the Interspeech COMPARE Challenge 2020. For the mask detection task, we train deep convolutional neural networks with filter-bank energies, gender-aware features, and speaker-aware features. Support Vector Machines follows as the back-end classifiers for binary prediction on the extracted deep embeddings. Several data augmentation schemes are used to increase the quantity of training data and improve our models' robustness, including speed perturbation, SpecAugment, and random erasing. For the speech breath monito…

2020-08-14cs.LGcs.SDeess.AS

Speech enhancement with mixture-of-deep-experts with clean clustering pre-training

Faculty of Engineering, Bar-Ilan University（巴伊兰大学工程学院）

In this study we present a mixture of deep experts (MoDE) neural-network architecture for single microphone speech enhancement. Our architecture comprises a set of deep neural networks (DNNs), each of which is an 'expert' in a different speech spectral pattern such as phoneme. A gating DNN is responsible for the latent variables which are the weights assigned to each expert's output given a speech segment. The experts estimate a mask from the noisy input and the final mask is then obtained as a weighted average of the experts' estimates, with the weights determined by the gating DNN. A soft sp…

2021-02-11cs.LGcs.SDeess.AS

Spatial and spectral deep attention fusion for multi-channel speech separation using deep embedding features

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）

Multi-channel deep clustering (MDC) has acquired a good performance for speech separation. However, MDC only applies the spatial features as the additional information. So it is difficult to learn mutual relationship between spatial and spectral features. Besides, the training objective of MDC is defined at embedding vectors, rather than real separated sources, which may damage the separation performance. In this work, we propose a deep attention fusion method to dynamically control the weights of the spectral and spatial features and combine them deeply. In addition, to solve the training obj…

2020-02-05cs.LGcs.SDeess.AS

基于延迟子带LSTM的在线单声道语音增强

Online Monaural Speech Enhancement Using Delayed Subband LSTM

Westlake University（西湖大学）

针对在线单声道语音增强需求，提出跨频率复用LSTM的延迟子带网络，通过子带训练与输出延迟前瞻架构降低参数量并提升性能，在DNS数据集上优于GRU全带基线方法。

2020-05-11cs.SDeess.AS

MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation

Sony Corporation（索尼公司）；Sony India Software Centre（索尼印度软件中心）

Deep neural networks have become an indispensable technique for audio source separation (ASS). It was recently reported that a variant of CNN architecture called MMDenseNet was successfully employed to solve the ASS problem of estimating source amplitudes, and state-of-the-art results were obtained for DSD100 dataset. To further enhance MMDenseNet, here we propose a novel architecture that integrates long short-term memory (LSTM) in multiple scales with skip connections to efficiently model long-term structures within an audio context. The experimental results show that the proposed method out…

2018-05-29cs.SDeess.AS

Real-time speech enhancement using equilibriated RNN

Waseda University（早稻田大学）；NTT Media Intelligence Laboratories（NTT媒体智能实验室）

We propose a speech enhancement method using a causal deep neural network~(DNN) for real-time applications. DNN has been widely used for estimating a time-frequency~(T-F) mask which enhances a speech signal. One popular DNN structure for that is a recurrent neural network~(RNN) owing to its capability of effectively modelling time-sequential data like speech. In particular, the long short-term memory (LSTM) is often used to alleviate the vanishing/exploding gradient problem which makes the training of an RNN difficult. However, the number of parameters of LSTM is increased as the price of miti…

2020-02-14cs.SDeess.AS