arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.19553cs.SD

超越学习长度:面向全流式文本语音合成的无训练长文本生成

Speaking Longer Than Learned: Training-Free Long-Form Generation for Full-Stream Text-to-Speech

University of Seoul(首尔大学)

Jeseung Lim, Jiseong Woo, Heeseung Kim

AI总结:

针对全流式TTS模型无法在无额外训练时生成长于训练长度文本的问题,通过分析VoXtream2的缺陷,提出两种无训练修改方案,实现长文本生成时的发音准确性、说话人相似度及自然度的保留。

摘要:

近期的全流式文本语音合成(TTS)模型可在输入文本到达时实现低延迟的个性化合成。然而,在不进行额外训练的情况下,将生成长度扩展至训练时所见长度之外的问题,在很大程度上尚未被探索。我们对VoXtream2进行了系统分析,该模型通过周期性丢弃过往上下文同时保留声学提示来扩展生成长度。我们发现该策略无法避免性能下降:在训练长度范围的末尾,发音准确性会因模型运行的位置范围而非过往上下文的丢失而恶化,而说话人相似度则在整个生成过程中逐渐下降。基于这些发现,我们提出两种无训练修改方案,即缩小注意力窗口并加强对声学提示的注意力。实验表明,这些修改方案在长文本生成过程中保持了发音准确性和说话人相似度,同时保留了原始模型的自然度。消融实验进一步显示了两种修改方案的互补效果。

英文摘要:

Recent full-stream text-to-speech (TTS) models enable low-latency personalized synthesis as input text arrives. However, sustaining generation beyond the lengths seen during training without additional training remains largely unexplored. We systematically analyze VoXtream2, which extends generation by periodically discarding past context while retaining the acoustic prompt. We find that this strategy does not prevent degradation: pronunciation accuracy deteriorates near the end of the training range due to the range of positions in which the model operates rather than the loss of past context, while speaker similarity gradually declines throughout generation. Based on these findings, we propose two training-free modifications that reduce the attention window and strengthen attention to the acoustic prompt. Experiments show that these modifications preserve pronunciation accuracy and speaker similarity throughout long-form generation while retaining the naturalness of the original model. Ablations further show complementary effects of the two modifications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑