FullSubNet is our recently proposed real-time single-channel speech enhancement network that achieves outstanding performance on the Deep Noise Suppression (DNS) Challenge dataset. A number of variants of FullSubNet have been proposed, but they all focus on the structure design towards better performance and are rarely concerned with computational efficiency. For many speech enhancement applications, a key feature is that system runs on a real-time, latency-sensitive, battery-powered platform, which strictly limits the algorithm latency and computational complexity. In this work, we propose a new architecture named Fast FullSubNet dedicated to accelerating the computation of FullSubNet. Specifically, Fast FullSubNet processes sub-band speech spectra in the mel-frequency domain by using cascaded linear-to-mel full-band, sub-band, and mel-to-linear full-band models such that frequencies involved in the sub-band computation are vastly reduced. After that, a down-sampling operation is proposed for the sub-band input sequence to further reduce the computational complexity along the time axis. Experimental results show that, compared to FullSubNet, Fast FullSubNet has only 13\% computational complexity and 16\% processing time, and achieves comparable or even better performance. Code and audio samples are available at https://github.com/Audio-WestlakeU/FullSubNet.
Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models
National Taiwan University(国立台湾大学);University of Washington(华盛顿大学);UC Berkeley(加州大学伯克利分校);Carnegie Mellon University(卡内基梅隆大学);NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)(国立台湾大学人工智能卓越研究中心)
Learning to Inference with Early Exit in the Progressive Speech Enhancement
Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);University of Chinese Academy of Sciences(中国科学院大学);Harbin Institute of Technology(哈尔滨工业大学)
Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
Seoul National University(首尔国立大学);Neosapience, Inc.(Neosapience公司)
虽然神经文本转语音(TTS)模型已经吸引了大量的关注,并成功地产生了类似人类的语音,但其自然度和架构效率仍有改进的空间。在这项工作中,我们提出了一种新的非自回归TTS模型,即Diff-TTS,它实现了高度自然和高效的语音合成。在给定文本的情况下,Diff-TTS利用去噪扩散框架通过扩散时间步长将噪声信号转换为梅尔频谱图。为了学习基于文本的梅尔语谱分布,我们提出了一种基于似然的优化方法。此外,为了提高推理速度,我们利用加速采样方法,使Diff-TTS生成原始波形更快,而不会显着降低感知质量。通过实验,我们验证了Diff-TTS的生成速度比使用单个NVIDIA 2080 Ti GPU的实时生成速度快28倍。
Optimizing Speech Multi-View Feature Fusion through Conditional Computation
Northeastern University(东北大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Meituan(美团);Huawei Translation Services Center(华为翻译服务中心);NiuTrans Research(NiuTrans研究中心)
SNIPER Training: Single-Shot Sparse Training for Text-to-Speech
Singapore University of Technology and Design(新加坡科技设计大学);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所);The University of Texas at Dallas(德克萨斯大学达拉斯分校)
Fast Multichannel Source Separation Based on Jointly Diagonalizable Spatial Covariance Matrices
Center for Advanced Intelligence Project (AIP), RIKEN(理化学研究所先进智能项目中心);Graduate School of Informatics, Kyoto University(京都大学信息学研究生院);National Institute of Advanced Industrial Science and Technology (AIST)(产业技术综合研究所)