VoXtream2:全流文本到语音系统与动态说话速率控制
VoXtream2: Full-stream TTS with dynamic speaking rate control
AI总结:
VoXtream2是一种全流文本到语音系统,通过动态控制说话速率实现低延迟和可控性,结合分布匹配机制和分类器自由引导提升合成质量,支持文本无关的音频提示。
摘要:
全流文本到语音(TTS)对于交互系统必须在最小延迟下启动说话,同时在文本逐步到达时保持可控。我们提出了VoXtream2,一种零样本全流TTS模型,具备动态说话速率控制能力,可在说话过程中实时更新。VoXtream2结合了持续状态的分布匹配机制和跨条件信号的分类器自由引导,以提高可控性和合成质量。提示文本遮蔽使文本无关的音频提示成为可能,无需提示转录。在标准零样本基准和专用说话速率测试集上,VoXtream2在较小模型和较少训练数据的情况下,实现了与公开基线竞争的客观和主观结果。在全流模式下,它在消费级GPU上以4倍于实时的速度运行,首包延迟为74毫秒。
英文摘要:
Full-stream text-to-speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtream2 combines a distribution matching mechanism over duration states with classifier-free guidance across conditioning signals to improve controllability and synthesis quality. Prompt-text masking enables textless audio prompting, removing the need for prompt transcription. Across standard zero-shot benchmarks and a dedicated speaking-rate test set, VoXtream2 achieves competitive objective and subjective results against public baselines despite a smaller model and less training data. In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。