Full-Duplex-Bench：评估全双工口语对话模型话轮转换能力的基准
Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
AI总结：
针对现有口语对话模型评估局限于半双工和粗粒度指标的问题，本文提出Full-Duplex-Bench基准，系统评估停顿处理、反馈通道、话轮转换和打断管理能力，并通过自动指标实现公平、快速、可复现的评估，以推动全双工口语对话模型的发展。
摘要：
口语对话建模带来的挑战超越了基于文本的语言建模，需要实时交互、话轮转换和反馈通道。虽然大多数口语对话模型（SDMs）以半双工模式运行——一次处理一个话轮——但新兴的全双工口语对话模型能够同时听和说，从而实现更自然的对话。然而，当前的评估仍然有限，主要集中于基于话轮的指标或粗粒度的语料库级分析。为了解决这一问题，我们引入了Full-Duplex-Bench，这是一个系统性地评估关键交互行为的基准：停顿处理、反馈通道、话轮转换和打断管理。我们的框架使用自动指标进行一致、可复现的评估，并提供了一个公平、快速的评估设置。通过发布我们的基准和代码，我们旨在推动口语对话建模的发展，并促进更自然、更具吸引力的口语对话模型的开发。
英文摘要：
Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (SDMs) operate in half-duplex mode-processing one turn at a time - emerging full-duplex SDMs can listen and speak simultaneously, enabling more natural conversations. However, current evaluations remain limited, focusing mainly on turn-based metrics or coarse corpus-level analyses. To address this, we introduce Full-Duplex-Bench, a benchmark that systematically evaluates key interactive behaviors: pause handling, backchanneling, turn-taking, and interruption management. Our framework uses automatic metrics for consistent, reproducible assessment and provides a fair, fast evaluation setup. By releasing our benchmark and code, we aim to advance spoken dialogue modeling and foster the development of more natural and engaging SDMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
