arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.02063cs.SDcs.AIeess.AS
收藏

SPMamba:语音分离中你所需的只是状态空间模型

SPMamba: State-space model is all you need in speech separation

Tsinghua University(清华大学);Chinese Institute for Brain Research (CIBR)(中国脑科学研究院)

Kai Li, Guo Chen, Runxuan Yang, Xiaolin Hu

AI总结:

本文提出SPMamba语音分离方法,在TF-GridNet中以双向Mamba替换BLSTM,线性复杂度建模长程时频依赖,并在多个数据集上实现更优性能与更低复杂度。

摘要:

现有的基于CNN的语音分离模型面临局部感受野限制,无法有效捕获长时间依赖关系。尽管基于LSTM和Transformer的语音分离模型可以避免这一问题,但其较高的复杂度使其在处理长音频时面临计算资源和推理效率方面的挑战。为应对这一挑战,我们提出了一种名为SPMamba的创新语音分离方法。该模型建立在稳健的TF-GridNet架构之上,用双向Mamba模块取代了其传统的BLSTM模块。这些模块可有效建模时间与频率维度之间的时空关系,使SPMamba能够以线性计算复杂度捕获长程依赖关系。具体而言,Mamba模块内部的双向处理使模型能够同时利用过去和未来的上下文信息,从而提升分离性能。在包括WSJ0-2Mix、WHAM!和Libri2Mix在内的公共数据集以及新构建的Echo2Mix数据集上进行的大量实验表明,SPMamba显著优于现有的最先进模型,在取得更优结果的同时降低了计算复杂度。这些发现凸显了SPMamba在应对复杂环境中语音分离复杂挑战方面的有效性。

英文摘要:

Existing CNN-based speech separation models face local receptive field limitations and cannot effectively capture long time dependencies. Although LSTM and Transformer-based speech separation models can avoid this problem, their high complexity makes them face the challenge of computational resources and inference efficiency when dealing with long audio. To address this challenge, we introduce an innovative speech separation method called SPMamba. This model builds upon the robust TF-GridNet architecture, replacing its traditional BLSTM modules with bidirectional Mamba modules. These modules effectively model the spatiotemporal relationships between the time and frequency dimensions, allowing SPMamba to capture long-range dependencies with linear computational complexity. Specifically, the bidirectional processing within the Mamba modules enables the model to utilize both past and future contextual information, thereby enhancing separation performance. Extensive experiments conducted on public datasets, including WSJ0-2Mix, WHAM!, and Libri2Mix, as well as the newly constructed Echo2Mix dataset, demonstrated that SPMamba significantly outperformed existing state-of-the-art models, achieving superior results while also reducing computational complexity. These findings highlighted the effectiveness of SPMamba in tackling the intricate challenges of speech separation in complex environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑