ASR基准测试：需要更具代表性的对话数据集
ASR Benchmarking: Need for a More Representative Conversational Dataset
AI总结：
针对现有ASR基准无法反映真实对话复杂性，本研究引入基于TalkBank的多语言对话数据集，发现SOTA模型性能显著下降，且词错误率与语音不流畅相关，凸显了构建更真实对话基准的必要性。
摘要：
自动语音识别（ASR）系统在广泛使用的基准测试（如LibriSpeech和Fleurs）上取得了显著性能。然而，这些基准未能充分反映真实世界对话环境的复杂性，其中语音通常是非结构化的，并包含停顿、打断和多样口音等不流畅现象。在本研究中，我们引入了一个多语言对话数据集，该数据集源自TalkBank，由成人之间的非结构化电话对话组成。我们的结果表明，在对话场景下测试时，各种最先进的ASR模型的性能显著下降。此外，我们观察到词错误率与语音不流畅现象的存在之间存在相关性，这凸显了对更真实、更具对话性的ASR基准测试的迫切需求。
英文摘要：
Automatic Speech Recognition (ASR) systems have achieved remarkable performance on widely used benchmarks such as LibriSpeech and Fleurs. However, these benchmarks do not adequately reflect the complexities of real-world conversational environments, where speech is often unstructured and contains disfluencies such as pauses, interruptions, and diverse accents. In this study, we introduce a multilingual conversational dataset, derived from TalkBank, consisting of unstructured phone conversation between adults. Our results show a significant performance drop across various state-of-the-art ASR models when tested in conversational settings. Furthermore, we observe a correlation between Word Error Rate and the presence of speech disfluencies, highlighting the critical need for more realistic, conversational ASR benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
