PersonaPlex：面向全双工对话语音模型的语音与角色控制
PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models
AI总结：
PersonaPlex通过融合角色条件和语音克隆技术，实现全双工对话语音模型的语音与角色控制，提升角色一致性、说话人相似性和对话自然度。
摘要：
近年来，全双工语音模型的进步使得自然、低延迟的语音到语音交互成为可能。然而，现有模型受限于固定角色和语音，限制了其在结构化、角色驱动的现实应用和个性化交互中的能力。在本工作中，我们引入了PersonaPlex，一种结合混合系统提示的全双工对话语音模型，融合角色条件和文本提示，以及语音克隆和语音样本。PersonaPlex是在一个大规模合成数据集上训练的，该数据集包含配对提示和用户-代理对话，通过开源大型语言模型（LLM）和文本到语音（TTS）模型生成。为了评估角色条件在现实场景中的表现，我们扩展了Full-Duplex-Bench基准测试，使其不仅限于单一助手角色，而是扩展到多角色客服场景。实验表明，PersonaPlex在角色条件行为、语音条件语音以及自然对话响应方面表现出色，超越了最先进的全双工语音模型和基于混合大型语言模型的语音系统，在角色一致性、说话人相似性、延迟和自然度方面均取得优势。
英文摘要：
Recent advances in duplex speech models have enabled natural, low-latency speech-to-speech interactions. However, existing models are restricted to a fixed role and voice, limiting their ability to support structured, role-driven real-world applications and personalized interactions. In this work, we introduce PersonaPlex, a duplex conversational speech model that incorporates hybrid system prompts, combining role conditioning with text prompts and voice cloning with speech samples. PersonaPlex is trained on a large-scale synthetic dataset of paired prompts and user-agent conversations, generated with open-source large language models (LLM) and text-to-speech (TTS) models. To evaluate role conditioning in real-world settings, we extend the Full-Duplex-Bench benchmark beyond a single assistant role to multi-role customer service scenarios. Experiments show that PersonaPlex achieves strong role-conditioned behavior, voice-conditioned speech, and natural conversational responsiveness, surpassing state-of-the-art duplex speech models and hybrid large language model-based speech systems in role adherence, speaker similarity, latency, and naturalness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
