arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.13518eess.AScs.CLcs.HCcs.LGcs.SD

VoXtream2:全流文本到语音系统与动态说话速率控制

VoXtream2: Full-stream TTS with dynamic speaking rate control

Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Sweden(语音、音乐与听觉系,皇家理工学院,瑞典)

Nikita Torgashov, Gustav Eje Henter, Gabriel Skantze

AI总结:

VoXtream2是一种全流文本到语音系统,通过动态控制说话速率实现低延迟和可控性,结合分布匹配机制和分类器自由引导提升合成质量,支持文本无关的音频提示。

摘要:

全流文本到语音(TTS)对于交互系统必须在最小延迟下启动说话,同时在文本逐步到达时保持可控。我们提出了VoXtream2,一种零样本全流TTS模型,具备动态说话速率控制能力,可在说话过程中实时更新。VoXtream2结合了持续状态的分布匹配机制和跨条件信号的分类器自由引导,以提高可控性和合成质量。提示文本遮蔽使文本无关的音频提示成为可能,无需提示转录。在标准零样本基准和专用说话速率测试集上,VoXtream2在较小模型和较少训练数据的情况下,实现了与公开基线竞争的客观和主观结果。在全流模式下,它在消费级GPU上以4倍于实时的速度运行,首包延迟为74毫秒。

英文摘要:

Full-stream text-to-speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtream2 combines a distribution matching mechanism over duration states with classifier-free guidance across conditioning signals to improve controllability and synthesis quality. Prompt-text masking enables textless audio prompting, removing the need for prompt transcription. Across standard zero-shot benchmarks and a dedicated speaking-rate test set, VoXtream2 achieves competitive objective and subjective results against public baselines despite a smaller model and less training data. In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑