FastTurn：统一声学和流式语义线索以实现低延迟和鲁棒的轮换检测
FastTurn: Unifying Acoustic and Streaming Semantic Cues for Low-Latency and Robust Turn Detection
AI总结：
FastTurn通过结合流式CTC解码与声学特征，实现低延迟和鲁棒的轮换检测，同时释放基于真实人类对话的测试集，提升实际全双工对话系统的性能。
摘要：
近期在AudioLLMs上的进展使口语对话系统能够超越基于轮换的交互，向实时全双工通信发展，其中代理必须在用户仍在说话时决定说话、让步或打断。现有全双工方法要么依赖于语音活动线索，缺乏语义理解，要么依赖于基于ASR的模块，引入延迟并在重叠语音和噪声下性能下降。此外，现有数据集很少捕捉真实的交互动态，限制了评估和部署。为缓解该问题，我们提出了FastTurn，一个统一的低延迟和鲁棒轮换检测框架。为了在保持性能的同时提高延迟，FastTurn结合了流式CTC解码与声学特征，使系统能够从部分观测中做出早期决策，同时保留语义线索。我们还发布了一个基于真实人类对话的测试集，捕捉了真实的轮换转换、重叠语音、反向通道、停顿、音调变化和环境噪声。实验表明，FastTurn在决策准确性和中断延迟方面优于代表性的基线方法，并在挑战性的声学条件下保持鲁棒性，证明了其在实际全双工对话系统中的有效性。
英文摘要：
Recent advances in AudioLLMs have enabled spoken dialogue systems to move beyond turn-based interaction toward real-time full-duplex communication, where the agent must decide when to speak, yield, or interrupt while the user is still talking. Existing full-duplex approaches either rely on voice activity cues, which lack semantic understanding, or on ASR-based modules, which introduce latency and degrade under overlapping speech and noise. Moreover, available datasets rarely capture realistic interaction dynamics, limiting evaluation and deployment. To mitigate the problem, we propose \textbf{FastTurn}, a unified framework for low-latency and robust turn detection. To advance latency while maintaining performance, FastTurn combines streaming CTC decoding with acoustic features, enabling early decisions from partial observations while preserving semantic cues. We also release a test set based on real human dialogue, capturing authentic turn transitions, overlapping speech, backchannels, pauses, pitch variation, and environmental noise. Experiments show FastTurn achieves higher decision accuracy with lower interruption latency than representative baselines and remains robust under challenging acoustic conditions, demonstrating its effectiveness for practical full-duplex dialogue systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
