arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.13985eess.AS
收藏

基于隐半马尔可夫模型结构化注意机制的神经序列到序列语音合成

Neural Sequence-to-Sequence Speech Synthesis Using a Hidden Semi-Markov Model Based Structured Attention Mechanism

Yoshihiko Nankaku, Kenta Sumiya, Takenori Yoshimura, Shinji Takaki, Kei Hashimoto, Keiichiro Oura, Keiichi Tokuda

摘要:

本文提出了一种新的序列到序列(Seq2Seq)模型,将隐半马尔可夫模型(HSMM)的结构集成到其注意力机制中。在语音合成中,已经表明,基于使用深度神经网络的Seq2Seq模型的方法可以在适当的条件下合成高质量的语音。然而,仍然存在几个基本问题,即,由于比对(两个序列之间的映射函数)的自由度过大而需要大量的训练数据,以及由于缺乏明确的持续时间建模而难以处理持续时间。该方法基于变分自动编码器(VAE)框架定义了一个生成模型,实现了对齐和模型参数的同时优化,并基于HSMM结构提供了单调对齐和显式持续时间建模。所提出的方法可以被视为使用Seq2Seq模型的基于隐马尔可夫模型(HMM)的语音合成和基于深度学习的语音合成的集成,结合了两者的益处。主观评价实验表明,该方法在相对少量的训练数据下获得了比Tacotron 2更高的平均意见分数。

英文摘要:

This paper proposes a novel Sequence-to-Sequence (Seq2Seq) model integrating the structure of Hidden Semi-Markov Models (HSMMs) into its attention mechanism. In speech synthesis, it has been shown that methods based on Seq2Seq models using deep neural networks can synthesize high quality speech under the appropriate conditions. However, several essential problems still have remained, i.e., requiring large amounts of training data due to an excessive degree for freedom in alignment (mapping function between two sequences), and the difficulty in handling duration due to the lack of explicit duration modeling. The proposed method defines a generative models to realize the simultaneous optimization of alignments and model parameters based on the Variational Auto-Encoder (VAE) framework, and provides monotonic alignments and explicit duration modeling based on the structure of HSMM. The proposed method can be regarded as an integration of Hidden Markov Model (HMM) based speech synthesis and deep learning based speech synthesis using Seq2Seq models, incorporating both the benefits. Subjective evaluation experiments showed that the proposed method obtained higher mean opinion scores than Tacotron 2 on relatively small amount of training data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑