VoiceStar：具有时长控制与外推能力的鲁棒零样本自回归语音合成
VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation
AI总结：
VoiceStar是首个支持输出时长控制与外推的零样本语音合成模型，通过PM-RoPE和CPM训练实现长语音生成，在短格式和长格式基准上均表现优异。
摘要：
我们提出了VoiceStar，这是第一个同时实现输出时长控制和时长外推的零样本语音合成模型。VoiceStar是一种自回归编码器-解码器神经编解码语言模型，利用了新颖的进度监控旋转位置嵌入（PM-RoPE），并通过连续提示混合（CPM）训练方法进行训练。PM-RoPE使模型能够更好地对齐文本和语音标记，指示生成语音的目标时长，并且还允许模型生成比训练期间所见时长长得多的语音波形。CPM训练还有助于缓解训练与推理之间的不匹配，并显著提高生成语音在说话人相似度和可懂度方面的质量。VoiceStar在Librispeech和Seed-TTS等短格式基准上优于或与当前最先进的模型相当，并且在长格式/外推基准（20-50秒）上在可懂度和自然度方面显著优于这些模型。代码和模型：https://github.com/jasonppy/VoiceStar。音频样本：https://jasonppy.github.io/VoiceStar_web
英文摘要：
We present VoiceStar, the first zero-shot TTS model that achieves both output duration control and extrapolation. VoiceStar is an autoregressive encoder-decoder neural codec language model, that leverages a novel Progress-Monitoring Rotary Position Embedding (PM-RoPE) and is trained with Continuation-Prompt Mixed (CPM) training. PM-RoPE enables the model to better align text and speech tokens, indicates the target duration for the generated speech, and also allows the model to generate speech waveforms much longer in duration than those seen during. CPM training also helps to mitigate the training/inference mismatch, and significantly improves the quality of the generated speech in terms of speaker similarity and intelligibility. VoiceStar outperforms or is on par with current state-of-the-art models on short-form benchmarks such as Librispeech and Seed-TTS, and significantly outperforms these models on long-form/extrapolation benchmarks (20-50s) in terms of intelligibility and naturalness. Code and models: https://github.com/jasonppy/VoiceStar. Audio samples: https://jasonppy.github.io/VoiceStar_web
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
