arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.04804eess.AScs.CLcs.LGcs.MM
收藏

AttentionStitch:注意力如何解决语音编辑问题

AttentionStitch: How Attention Solves the Speech Editing Problem

University of California Irvine(加州大学欧文分校)

Antonios Alexos, Pierre Baldi

AI总结:

针对语音编辑需无缝整合编辑内容的需求,提出基于预训练TTS模型加双注意力块的AttentionStitch模型,在多数据集上经主客观评估表现优于现有基线,可自动生成高质量、处理未见过词且速度快的编辑语音。

摘要:

从文本生成自然且高质量的语音是自然语言处理领域的一项挑战性问题。除语音生成外,语音编辑也是一项关键任务,要求将编辑后的语音无缝且不易察觉地整合到合成语音中。我们提出了一种新颖的语音编辑方法,利用预训练的文本转语音(TTS)模型(如FastSpeech 2),并在其之上加入双注意力块网络,自动将合成梅尔频谱图与编辑后文本的梅尔频谱图融合。我们将该模型命名为AttentionStitch,因为它利用注意力机制将音频样本拼接在一起。我们在单说话人和多说话人数据集(即LJSpeech和VCTK)上,将所提出的AttentionStitch模型与当前最优基线进行对比评估。通过包含15名人类参与者的客观和主观评估测试,我们证明了其优越性能。AttentionStitch能够生成高质量语音,即使是训练期间未见过的单词也能处理,同时无需人工干预即可自动运行。此外,AttentionStitch在训练和推理阶段速度都很快,能够生成听感自然的编辑后语音。

英文摘要:

The generation of natural and high-quality speech from text is a challenging problem in the field of natural language processing. In addition to speech generation, speech editing is also a crucial task, which requires the seamless and unnoticeable integration of edited speech into synthesized speech. We propose a novel approach to speech editing by leveraging a pre-trained text-to-speech (TTS) model, such as FastSpeech 2, and incorporating a double attention block network on top of it to automatically merge the synthesized mel-spectrogram with the mel-spectrogram of the edited text. We refer to this model as AttentionStitch, as it harnesses attention to stitch audio samples together. We evaluate the proposed AttentionStitch model against state-of-the-art baselines on both single and multi-speaker datasets, namely LJSpeech and VCTK. We demonstrate its superior performance through an objective and a subjective evaluation test involving 15 human participants. AttentionStitch is capable of producing high-quality speech, even for words not seen during training, while operating automatically without the need for human intervention. Moreover, AttentionStitch is fast during both training and inference and is able to generate human-sounding edited speech.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑