arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09785cs.CL
收藏

在巴马拉语言自动语音识别领域我们处于什么位置?

Where Are We At with Automatic Speech Recognition for the Bambara Language?

MALIBA-AI;RobotsMali AI;Rochester Institute of Technology(罗切斯特理工学院);DJELIA;Dakar American University of Science and Technology(达喀尔美国科学与技术大学)

Seydou Diallo, Yacouba Diarra, Mamadou K. Keita, Panga Azazia Kamaté, Adam Bouno Kampo, Aboubacar Ouattara

AI总结:

本文提出首个巴马拉语言ASR标准化基准,评估37种模型后发现其性能远低于部署标准,多语言预训练和模型扩展不足以提升少数语言表现。

摘要:

本文介绍了首个用于评估巴马拉语言自动语音识别(ASR)的标准化基准,利用一小时的专业录制马里宪法文本。该基准被设计为在接近最优的声学和语言条件下进行受控参考集,用于评估37个模型,从训练巴马拉语言的系统到大规模商用模型。我们的发现表明,当前ASR性能在狭窄的正式领域内仍显著低于部署标准;在词错误率(WER)方面表现最好的系统达到了46.76%,而另一个系统在字符错误率(CER)方面达到了13.00%的最佳成绩,而几个主要的多语言模型则超过了100%的WER。这些结果表明,仅依靠多语言预训练和模型扩展对于少数语言来说是不够的。此外,因为这个数据集代表了最简化和正式的口语巴马拉语的最佳情况,这些数字尚未在实际、真实世界环境中得到测试。我们提供了该基准和一个配套的公开排行榜,以促进透明的评估和未来巴马拉语语音技术的研究。

英文摘要:

This paper introduces the first standardized benchmark for evaluating Automatic Speech Recognition (ASR) in the Bambara language, utilizing one hour of professionally recorded Malian constitutional text. Designed as a controlled reference set under near-optimal acoustic and linguistic conditions, the benchmark was used to evaluate 37 models, ranging from Bambara-trained systems to large-scale commercial models. Our findings reveal that current ASR performance remains significantly below deployment standards in a narrow formal domain; the top-performing system in terms of Word Error Rate (WER) achieved 46.76\% and the best Character Error Rate (CER) of 13.00\% was set by another model, while several prominent multilingual models exceeded 100\% WER. These results suggest that multilingual pre-training and model scaling alone are insufficient for underrepresented languages. Furthermore, because this dataset represents a best-case scenario of the most simplified and formal form of spoken Bambara, these figures are yet to be tested against practical, real-world settings. We provide the benchmark and an accompanying public leaderboard to facilitate transparent evaluation and future research in Bambara speech technology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑