WavBench: 语音对话模型推理、口语化和副语言能力的基准测试
WavBench: Benchmarking Reasoning, Colloquialism, and Paralinguistics for End-to-End Spoken Dialogue Models
AI总结：
WavBench通过三重框架评估语音对话模型的推理、口语化和副语言能力，推动真实场景下的对话模型发展。
摘要：
随着高级推理能力的快速整合进入语音对话模型，该领域迫切需要能够超越简单交互、解决现实复杂性的基准测试。然而，当前的评估主要遵循文本生成标准，忽视了副语言和口语化独特的音频中心特性，以及现代代理所需的认知深度。为填补这一差距，我们引入WavBench，一个全面的基准测试，旨在评估现实对话能力，而先前的工作在此方面有所不足。独特的是，WavBench建立了一个三位一体的框架：1）Pro子集，旨在严格挑战增强推理的模型，显著增加难度；2）基础子集，定义了一个新的口语化标准，通过自然词汇、语言流畅性和互动关系优先考虑“可听性”，而不是严格的书面准确性；3）声学子集，涵盖显性理解、生成和隐性对话，严格评估在真实世界场景中的综合副语言能力。通过评估五个最先进的模型，WavBench提供了对复杂问题解决、口语化交付和副语言忠实度交集的关键见解，指导稳健语音对话模型的发展。基准数据集和评估工具包可在https://naruto-2024.github.io/wavbench.github.io/上获取。
英文摘要：
With the rapid integration of advanced reasoning capabilities into spoken dialogue models, the field urgently demands benchmarks that transcend simple interactions to address real-world complexity. However, current evaluations predominantly adhere to text-generation standards, overlooking the unique audio-centric characteristics of paralinguistics and colloquialisms, alongside the cognitive depth required by modern agents. To bridge this gap, we introduce WavBench, a comprehensive benchmark designed to evaluate realistic conversational abilities where prior works fall short. Uniquely, WavBench establishes a tripartite framework: 1) Pro subset, designed to rigorously challenge reasoning-enhanced models with significantly increased difficulty; 2) Basic subset, defining a novel standard for spoken colloquialism that prioritizes "listenability" through natural vocabulary, linguistic fluency, and interactive rapport, rather than rigid written accuracy; and 3) Acoustic subset, covering explicit understanding, generation, and implicit dialogue to rigorously evaluate comprehensive paralinguistic capabilities within authentic real-world scenarios. Through evaluating five state-of-the-art models, WavBench offers critical insights into the intersection of complex problem-solving, colloquial delivery, and paralinguistic fidelity, guiding the evolution of robust spoken dialogue models. The benchmark dataset and evaluation toolkit are available at https://naruto-2024.github.io/wavbench.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
