FlashLabs Chroma 1.0: 一种实时端到端语音对话模型及个性化语音克隆
FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloning
AI总结：
FlashLabs Chroma 1.0是一种实时端到端语音对话模型，通过交错调度实现低延迟和高保真个性化语音克隆，提升了说话人相似性并保持对话能力。
摘要：
最近的端到端语音对话系统利用语音分词器和神经音频编解码器，使LLM能够直接在离散的语音表示上操作。然而，这些模型往往表现出有限的说话人身份保持能力，阻碍了个性化语音交互。在本工作中，我们提出了Chroma 1.0，首个开源、实时、端到端语音对话模型，实现了低延迟交互和高保真的个性化语音克隆。Chroma通过交错的文本-音频标记调度（1:2）实现了亚秒级端到端延迟，同时在多轮对话中保持高质量的个性化语音合成。我们的实验结果表明，Chroma在人类基线上的说话人相似性提高了10.96%，实时因子（RTF）为0.43，同时保持了强大的推理和对话能力。我们的代码和模型可在https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma和https://huggingface.co/FlashLabs/Chroma-4B公开获取。
英文摘要：
Recent end-to-end spoken dialogue systems leverage speech tokenizers and neural audio codecs to enable LLMs to operate directly on discrete speech representations. However, these models often exhibit limited speaker identity preservation, hindering personalized voice interaction. In this work, we present Chroma 1.0, the first open-source, real-time, end-to-end spoken dialogue model that achieves both low-latency interaction and high-fidelity personalized voice cloning. Chroma achieves sub-second end-to-end latency through an interleaved text-audio token schedule (1:2) that supports streaming generation, while maintaining high-quality personalized voice synthesis across multi-turn conversations. Our experimental results demonstrate that Chroma achieves a 10.96% relative improvement in speaker similarity over the human baseline, with a Real-Time Factor (RTF) of 0.43, while maintaining strong reasoning and dialogue capabilities. Our code and models are publicly available at https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma and https://huggingface.co/FlashLabs/Chroma-4B .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
