arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.09337eess.AScs.AIcs.SD
收藏

Wave-U-Mamba:一个用于高质量且高效语音超分辨率的端到端框架

Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution

Korea University(高丽大学)

Yongjoon Lee, Chanwoo Kim

AI总结:

针对语音超分辨率中梅尔特征缺乏相位信息导致性能下降的问题,提出端到端时域框架 Wave-U-Mamba,基于选择性状态空间模型在 8-24 kHz 采样率下取得最低 LSD,生成自然音质且速度比基线快九倍以上、参数量不到 2%。

摘要:

语音超分辨率(SSR)是一项通过恢复缺失的高频成分来增强低分辨率语音信号的任务。传统方法通常重建对数梅尔特征,随后通过声码器在波形域生成高分辨率语音。然而,由于梅尔特征缺乏相位信息,这可能导致重建阶段的性能下降。受选择性状态空间模型(SSMs)近期进展的启发,我们提出了一种名为 Wave-U-Mamba 的方法,直接在时域执行 SSR。在我们的比较研究中,包括 WSRGlow、NU-Wave 2 和 AudioSR 等模型,Wave-U-Mamba 展现出卓越的性能,在 8 至 24 kHz 的各种低分辨率采样率下,实现了最低的对数谱距离(LSD)。此外,使用平均意见分(MOS)评分的主观人类评估表明,我们的方法生成的 SSR 具有自然且类似人类的音质。进一步地,Wave-U-Mamba 在取得这些结果的同时,在单张 A100 GPU 上生成高分辨率语音的速度比基线模型快九倍以上,且参数规模不到基线模型的 2%。

英文摘要:

Speech Super-Resolution (SSR) is a task of enhancing low-resolution speech signals by restoring missing high-frequency components. Conventional approaches typically reconstruct log-mel features, followed by a vocoder that generates high-resolution speech in the waveform domain. However, as mel features lack phase information, this can result in performance degradation during the reconstruction phase. Motivated by recent advances with Selective State Spaces Models (SSMs), we propose a method, referred to as Wave-U-Mamba that directly performs SSR in time domain. In our comparative study, including models such as WSRGlow, NU-Wave 2, and AudioSR, Wave-U-Mamba demonstrates superior performance, achieving the lowest Log-Spectral Distance (LSD) across various low-resolution sampling rates, ranging from 8 to 24 kHz. Additionally, subjective human evaluations, scored using Mean Opinion Score (MOS) reveal that our method produces SSR with natural and human-like quality. Furthermore, Wave-U-Mamba achieves these results while generating high-resolution speech over nine times faster than baseline models on a single A100 GPU, with parameter sizes less than 2\% of those in the baseline models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑