arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.20997cs.SDcs.LGeess.AS

SepMamba:使用 Mamba 进行说话人分离的状态空间模型

SepMamba: State-space models for speaker separation using Mamba

Technical University of Denmark(丹麦技术大学);WS Audiology A/S(WS听力科技公司)

Thor Højhus Avenstrup, Boldizsár Elek, István László Mádi, András Bence Schin, Morten Mørup, Bjørn Sand Jensen, Kenny Falkær Olsen

AI总结:

本文提出主要由双向 Mamba 层构成的 U-Net 架构 SepMamba,用于单通道说话人分离,并在 WSJ0 2-speaker 上以更低计算、内存和前向时间超过同规模主流模型。

摘要:

近年来,基于深度学习的单通道说话人分离性能显著提升,这主要得益于基于 Transformer 的注意力机制的引入。然而,这些提升以高昂的计算需求为代价,使其无法用于许多实际应用。作为具有相似建模能力且计算高效的替代方案,Mamba 最近被提出。我们提出 SepMamba,这是一种主要由双向 Mamba 层组成的基于 U-Net 的架构。我们发现,在 WSJ0 2-speaker 数据集上,我们的方法优于包括基于 Transformer 的模型在内的类似规模的知名模型,同时显著降低了计算成本、内存使用量和前向传播时间。我们还报告了 SepMamba 因果变体的出色结果。对于深度语音分离而言,我们的方法为基于 Transformer 的架构提供了一种在计算上更具优势的替代方案。

英文摘要:

Deep learning-based single-channel speaker separation has improved significantly in recent years largely due to the introduction of the transformer-based attention mechanism. However, these improvements come at the expense of intense computational demands, precluding their use in many practical applications. As a computationally efficient alternative with similar modeling capabilities, Mamba was recently introduced. We propose SepMamba, a U-Net-based architecture composed primarily of bidirectional Mamba layers. We find that our approach outperforms similarly-sized prominent models - including transformer-based models - on the WSJ0 2-speaker dataset while enjoying a significant reduction in computational cost, memory usage, and forward pass time. We additionally report strong results for causal variants of SepMamba. Our approach provides a computationally favorable alternative to transformer-based architectures for deep speech separation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑