AfriSpeech-MultiBench：面向非洲口音英语自动语音识别的垂直化多领域多国基准套件
AfriSpeech-MultiBench: A Verticalized Multidomain Multicountry Benchmark Suite for African Accented English ASR
AI总结：
针对现有语音模型评估未覆盖非洲语言多样性的缺口，提出首个多领域多国非洲口音英语ASR基准AfriSpeech-MultiBench，通过多类系统测试揭示性能差异，支撑适配非洲场景的包容性语音技术选型。
摘要：
包括Google NotebookLM、OpenAI语音转语音API在内的语音赋能AI近期进展，正推动全球范围内对语音接口的广泛关注。尽管发展势头迅猛，目前尚无适配非洲语言多样性的公开特定应用模型评估方案。我们推出AfriSpeech-MultiBench，这是首个面向10余个国家、覆盖金融、法律、医疗、通用对话、呼叫中心、命名实体、幻觉鲁棒性七大应用领域、超100种非洲英语口音的领域专属评估套件。我们利用来自多个公开非洲口音英语语音数据集的即兴与非即兴语音对话，对各类开源、闭源、单模态ASR以及基于多模态大语言模型的语音识别系统开展基准测试。实证分析显示存在系统性差异：开源ASR模型在即兴语音场景表现优异，但在带噪声的非母语对话上性能下降；多模态大语言模型口音鲁棒性更强，但难以处理领域专属命名实体；专有模型在干净语音上准确率高，但表现随国家和领域差异波动显著。经非洲英语微调的模型可在更低延迟下达到有竞争力的准确率，具备部署实用优势，但幻觉仍是多数SOTA模型的突出问题。通过发布这一综合基准，我们助力从业者与研究者选择适配非洲场景的语音技术，推动为服务不足社区构建包容性语音应用。
英文摘要：
Recent advances in speech-enabled AI, including Google's NotebookLM and OpenAI's speech-to-speech API, are driving widespread interest in voice interfaces globally. Despite this momentum, there exists no publicly available application-specific model evaluation that caters to Africa's linguistic diversity. We present AfriSpeech-MultiBench, the first domain-specific evaluation suite for over 100 African English accents across 10+ countries and seven application domains: Finance, Legal, Medical, General dialogue, Call Center, Named Entities and Hallucination Robustness. We benchmark a diverse range of open, closed, unimodal ASR and multimodal LLM-based speech recognition systems using both spontaneous and non-spontaneous speech conversation drawn from various open African accented English speech datasets. Our empirical analysis reveals systematic variation: open-source ASR models excels in spontaneous speech contexts but degrades on noisy, non-native dialogue; multimodal LLMs are more accent-robust yet struggle with domain-specific named entities; proprietary models deliver high accuracy on clean speech but vary significantly by country and domain. Models fine-tuned on African English achieve competitive accuracy with lower latency, a practical advantage for deployment, hallucinations still remain a big problem for most SOTA models. By releasing this comprehensive benchmark, we empower practitioners and researchers to select voice technologies suited to African use-cases, fostering inclusive voice applications for underserved communities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
