arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.16808cs.SDeess.AS
收藏

探索Mamba在语音应用中的能力

Exploring the Capability of Mamba in Speech Applications

CyberAgent, Inc.(赛博代理公司);Tokyo Metropolitan University(东京都立大学)

Koichi Miyazaki, Yoshiki Masuyama, Masato Murata

AI总结:

本研究探索基于状态空间模型的Mamba架构在语音领域的应用潜力,通过在自动语音识别、文本转语音等多类语音任务上与主流Transformer变体对比,验证了其性能可比肩甚至更优,且在长语音处理中具备效率优势。

摘要:

本文探索了Mamba——一种近期提出的基于状态空间模型(SSMs)的架构——作为基于Transformer的模型的有力替代方案的能力。在语音领域,设计精良的基于Transformer的模型(如Conformer和E-Branchformer)已成为事实上的标准。大量评估已证明这些基于Transformer的模型在广泛语音任务中的有效性。相比之下,对SSMs的评估仅局限于少数任务,如自动语音识别(ASR)和语音合成。本文中,我们将Mamba与最先进的Transformer变体在多种语音应用中进行了对比,包括ASR、文本转语音、口语理解和语音摘要。实验评估显示,Mamba取得了与基于Transformer的模型相当甚至更优的性能,并展现出其在长语音处理中的效率。

英文摘要:

This paper explores the capability of Mamba, a recently proposed architecture based on state space models (SSMs), as a competitive alternative to Transformer-based models. In the speech domain, well-designed Transformer-based models, such as the Conformer and E-Branchformer, have become the de facto standards. Extensive evaluations have demonstrated the effectiveness of these Transformer-based models across a wide range of speech tasks. In contrast, the evaluation of SSMs has been limited to a few tasks, such as automatic speech recognition (ASR) and speech synthesis. In this paper, we compared Mamba with state-of-the-art Transformer variants for various speech applications, including ASR, text-to-speech, spoken language understanding, and speech summarization. Experimental evaluations revealed that Mamba achieves comparable or better performance than Transformer-based models, and demonstrated its efficiency in long-form speech processing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑