Easy Turn：融合声学与语言模态以实现全双工口语对话系统中的稳健话轮转换
Easy Turn: Integrating Acoustic and Linguistic Modalities for Robust Turn-Taking in Full-Duplex Spoken Dialogue Systems
AI总结：
针对全双工对话中话轮转换检测依赖闭源、大模型或单模态的问题，提出开源双模态 Easy Turn 模型及 1,145 小时数据集，在开源测试集上达到最优准确率。
摘要：
全双工交互对于自然人机通信至关重要，但它仍具有挑战性，因为系统需要稳健的话轮转换检测，以决定何时说话、倾听或保持沉默。现有解决方案要么依赖专用话轮转换模型，而其中大多数尚未开源；少数可用模型又受限于参数量庞大，或仅支持声学或语言等单一模态。另一些方法通过微调 LLM 主干来赋予全双工能力，但这需要大量全双工数据，而此类数据在开源领域仍然稀缺。为解决这些问题，我们提出 Easy Turn，这是一个开源、模块化的话轮转换检测模型，融合声学与语言双模态信息，用于预测四种对话话轮状态：complete、incomplete、backchannel 和 wait；同时发布 Easy Turn trainset，这是一个用于训练话轮转换检测模型的 1,145 小时语音数据集。与 TEN Turn Detection 和 Smart Turn V2 等现有开源模型相比，我们的模型在开源 Easy Turn testset 上取得了最先进的话轮转换检测准确率。数据和模型将在 GitHub 上公开发布。
英文摘要：
Full-duplex interaction is crucial for natural human-machine communication, yet remains challenging as it requires robust turn-taking detection to decide when the system should speak, listen, or remain silent. Existing solutions either rely on dedicated turn-taking models, most of which are not open-sourced. The few available ones are limited by their large parameter size or by supporting only a single modality, such as acoustic or linguistic. Alternatively, some approaches finetune LLM backbones to enable full-duplex capability, but this requires large amounts of full-duplex data, which remain scarce in open-source form. To address these issues, we propose Easy Turn, an open-source, modular turn-taking detection model that integrates acoustic and linguistic bimodal information to predict four dialogue turn states: complete, incomplete, backchannel, and wait, accompanied by the release of Easy Turn trainset, a 1,145-hour speech dataset designed for training turn-taking detection models. Compared to existing open-source models like TEN Turn Detection and Smart Turn V2, our model achieves state-of-the-art turn-taking detection accuracy on our open-source Easy Turn testset. The data and model will be made publicly available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
