arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.09768eess.AScs.SD
收藏

基于大规模离散Token的语音语言模型的文本转语音合成评估

Evaluating Text-to-Speech Synthesis from a Large Discrete Token-based Speech Language Model

KTH Royal Institute of Technology(瑞典皇家理工学院)

Siyang Wang, Éva Székely

AI总结:

该研究从五个维度评估基于离散token的语音语言模型的文本转语音效果,发现其韵律多样性等表现优于传统TTS,但可懂度等存在不足,扩大模型规模可小幅提升鲁棒性,为相关研究提供基准。

摘要:

将生成式语言建模应用于离散语音token的最新进展为文本转语音(TTS)合成开辟了新途径。这类语音语言模型(SLMs)与文本语言模型类似,具备可扩展性、概率性和上下文感知能力。尽管它们能生成多样且自然的输出,但有时会出现可懂度不足、混入非语音噪声或产生幻觉等问题。随着这一创新范式在语音合成领域的应用日益广泛,显然有必要深入评估其能力与局限性。本文通过自动指标和听力测试,对基于离散token的SLM的TTS效果展开评估,考察了五个关键维度:说话风格、可懂度、说话人一致性、韵律变化、自发行为。结果表明,该模型在生成多样韵律和自发输出方面表现突出,在听力测试中,其自然度和语境适宜性评分也高于传统TTS。不过,该模型在可懂度和说话人一致性上的表现落后于传统TTS。此外,研究显示扩大SLMs的规模能小幅提升鲁棒性。本研究结果旨在为语音合成领域生成式SLMs的未来发展提供基准。

英文摘要:

Recent advances in generative language modeling applied to discrete speech tokens presented a new avenue for text-to-speech (TTS) synthesis. These speech language models (SLMs), similarly to their textual counterparts, are scalable, probabilistic, and context-aware. While they can produce diverse and natural outputs, they sometimes face issues such as unintelligibility and the inclusion of non-speech noises or hallucination. As the adoption of this innovative paradigm in speech synthesis increases, there is a clear need for an in-depth evaluation of its capabilities and limitations. In this paper, we evaluate TTS from a discrete token-based SLM, through both automatic metrics and listening tests. We examine five key dimensions: speaking style, intelligibility, speaker consistency, prosodic variation, spontaneous behaviour. Our results highlight the model's strength in generating varied prosody and spontaneous outputs. It is also rated higher in naturalness and context appropriateness in listening tests compared to a conventional TTS. However, the model's performance in intelligibility and speaker consistency lags behind traditional TTS. Additionally, we show that increasing the scale of SLMs offers a modest boost in robustness. Our findings aim to serve as a benchmark for future advancements in generative SLMs for speech synthesis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑