StreamMel：基于交错连续自回归建模的实时零样本文本转语音
StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling
AI总结：
针对现有零样本TTS多为离线设计、流式方案性能欠佳的问题，提出单阶段流式框架StreamMel，通过文本与声学帧交错的连续自回归建模实现低延迟高质量合成，性能媲美离线系统，适配实时语音大模型集成需求。
摘要：
零样本文本转语音（TTS）合成的最新进展已能为未见过的说话人生成高质量语音，但大多数系统因采用离线设计，仍不适用于实时应用。当前流式TTS范式通常依赖多阶段流水线和离散表示，导致计算成本上升且系统性能欠佳。本文提出StreamMel，这是首个对连续梅尔频谱图建模的单阶段流式TTS框架。通过将文本标记与声学帧交错排列，StreamMel可实现低延迟自回归合成，同时保持高说话人相似度和自然度。在LibriSpeech上的实验表明，StreamMel在质量和延迟方面均优于现有流式TTS基线，其性能甚至可与离线系统媲美，同时支持高效实时生成，在与实时语音大语言模型集成方面展现出广阔前景。音频样本可访问：https://aka.ms/StreamMel。
英文摘要：
Recent advances in zero-shot text-to-speech (TTS) synthesis have achieved high-quality speech generation for unseen speakers, but most systems remain unsuitable for real-time applications because of their offline design. Current streaming TTS paradigms often rely on multi-stage pipelines and discrete representations, leading to increased computational cost and suboptimal system performance. In this work, we propose StreamMel, a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms. By interleaving text tokens with acoustic frames, StreamMel enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness. Experiments on LibriSpeech demonstrate that StreamMel outperforms existing streaming TTS baselines in both quality and latency. It even achieves performance comparable to offline systems while supporting efficient real-time generation, showcasing broad prospects for integration with real-time speech large language models. Audio samples are available at: https://aka.ms/StreamMel.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
