FireRedChat：一种可插拔的全双工语音交互系统，具有级联和半级联实现
FireRedChat: A Pluggable, Full-Duplex Voice Interaction System with Cascaded and Semi-Cascaded Implementations
AI总结：
FireRedChat提出了一种可插拔的全双工语音交互系统，通过集成流式pVAD和语义结束检测的控制器，将级联和半级联流水线升级为全双工，实现了低延迟、高鲁棒性的自然交互。
摘要：
全双工语音交互允许用户和智能体在可控插话（barge-in）的情况下同时说话，从而实现逼真的助手和客户服务。现有解决方案要么是端到端的，难以设计且难以控制，要么是由轮流说话控制器（turn-taking controllers）管理的模块化流水线，便于升级和逐模块优化；然而，先前的模块化框架依赖于非开源组件和外部提供商，限制了整体优化。在这项工作中，我们提出了一个完整、实用的全双工语音交互系统，包括轮流说话控制器、交互模块和对话管理器。该控制器集成了流式个性化语音活动检测（pVAD），以抑制来自噪声和非主要说话者的虚假插话，精确标记主要说话者片段的时间戳，并明确启用主要说话者的插话；语义性话语结束检测器改进了停止决策。它将异构的半双工流水线（级联、半级联和语音到语音）升级为全双工。使用内部模型，我们实现了级联和半级联变体；半级联变体捕获情感和副语言线索，产生更连贯的响应，降低延迟和错误传播，并提高鲁棒性。对话管理器通过工具调用和上下文管理扩展了能力。我们还提出了三个系统级指标：插话（barge-in）、话语结束检测准确率和端到端延迟，以评估自然度、控制准确性和效率。实验表明，虚假中断更少，语义结束更准确，延迟更低，接近工业系统，从而实现鲁棒、自然、实时的全双工交互。演示：https://fireredteam.github.io/demos/firered_chat。
英文摘要：
Full-duplex voice interaction allows users and agents to speak simultaneously with controllable barge-in, enabling lifelike assistants and customer service. Existing solutions are either end-to-end, difficult to design and hard to control, or modular pipelines governed by turn-taking controllers that ease upgrades and per-module optimization; however, prior modular frameworks depend on non-open components and external providers, limiting holistic optimization. In this work, we present a complete, practical full-duplex voice interaction system comprising a turn-taking controller, an interaction module, and a dialogue manager. The controller integrates streaming personalized VAD (pVAD) to suppress false barge-ins from noise and non-primary speakers, precisely timestamp primary-speaker segments, and explicitly enable primary-speaker barge-ins; a semantic end-of-turn detector improves stop decisions. It upgrades heterogeneous half-duplex pipelines, cascaded, semi-cascaded, and speech-to-speech, to full duplex. Using internal models, we implement cascaded and semi-cascaded variants; the semi-cascaded one captures emotional and paralinguistic cues, yields more coherent responses, lowers latency and error propagation, and improves robustness. A dialogue manager extends capabilities via tool invocation and context management. We also propose three system-level metrics, barge-in, end-of-turn detection accuracy, and end-to-end latency, to assess naturalness, control accuracy, and efficiency. Experiments show fewer false interruptions, more accurate semantic ends, and lower latency approaching industrial systems, enabling robust, natural, real-time full-duplex interaction. Demos: https://fireredteam.github.io/demos/firered_chat.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
