TacoLM:配备门控注意力的编解码器语言模型是高效的零样本文本转语音合成器
TacoLM: GaTed Attention Equipped Codec Language Model are Efficient Zero-Shot Text to Speech Synthesizers
AI总结:
针对神经编解码器语言模型在零样本TTS中推理慢、稳定性不足的问题,提出带门控注意力与门控交叉注意力的TacoLM,相比VALL-E参数量减90%、速度提5.2倍且语音质量更优。
摘要:
神经编解码器语言模型(LM)已在零样本文本转语音(TTS)合成中展现出强大能力。然而,由于其自回归特性以及文本与音频之间的隐式对齐,编解码器LM常存在推理速度和稳定性方面的局限。本研究中,为应对这些挑战,我们提出了一种神经编解码器LM的新变体,即TacoLM。具体而言,TacoLM引入门控注意力机制以提升训练和推理效率并减小模型规模。同时,每个解码器层都增设了一个门控交叉注意力层,可提升合成语音的效率和内容准确性。在Librispeech语料库上的评估中,与VALL-E相比,所提出的TacoLM取得了更优的词错误率、说话人相似度和平均意见得分,且参数量减少90%,速度提升5.2倍。演示和代码可在https://ereboas.github.io/TacoLM/获取。
英文摘要:
Neural codec language model (LM) has demonstrated strong capability in zero-shot text-to-speech (TTS) synthesis. However, the codec LM often suffers from limitations in inference speed and stability, due to its auto-regressive nature and implicit alignment between text and audio. In this work, to handle these challenges, we introduce a new variant of neural codec LM, namely TacoLM. Specifically, TacoLM introduces a gated attention mechanism to improve the training and inference efficiency and reduce the model size. Meanwhile, an additional gated cross-attention layer is included for each decoder layer, which improves the efficiency and content accuracy of the synthesized speech. In the evaluation of the Librispeech corpus, the proposed TacoLM achieves a better word error rate, speaker similarity, and mean opinion score, with 90% fewer parameters and 5.2 times speed up, compared with VALL-E. Demo and code is available at https://ereboas.github.io/TacoLM/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。