未被倾听的声音：约鲁巴语地区方言的NLP资源与模型
Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects
AI总结：
本文构建了覆盖四个约鲁巴语地区方言的高质量平行文本与语音语料库YORÙLECT，并通过机器翻译、语音识别和语音翻译实验揭示方言性能差距，证明方言自适应微调可有效缩小该差距。
摘要：
约鲁巴语是一种拥有约4700万使用者的非洲语言，包含一个由多种方言构成的连续体。近期为非洲语言开发NLP技术的努力主要集中在标准方言上，导致那些资源或工具极少甚至缺失的方言和变体之间存在差距。我们通过引入一个跨越三个领域和四种约鲁巴语地区方言的新型高质量平行文本与语音语料库YORÙLECT，朝着弥合这一差距迈出了步伐。为了构建该语料库，我们邀请了母语者参与，前往使用这些方言的社区采集文本和语音数据。利用我们新建的语料库，我们在（文本）机器翻译、自动语音识别和语音到文本翻译上开展了大量实验。我们的结果揭示了标准约鲁巴语与其他方言在所有任务上存在显著的性能差距。然而，我们也表明，通过方言自适应微调，我们能够缩小这一差距。我们相信，我们的数据集和实验分析将通过加深对现有挑战的理解，并为后续开发提供高质量数据集，极大地推动约鲁巴语及其方言乃至其他非洲语言的NLP工具开发。我们在开放许可下公开发布YORÙLECT数据集和模型。
英文摘要：
Yorùbá an African language with roughly 47 million speakers encompasses a continuum with several dialects. Recent efforts to develop NLP technologies for African languages have focused on their standard dialects, resulting in disparities for dialects and varieties for which there are little to no resources or tools. We take steps towards bridging this gap by introducing a new high-quality parallel text and speech corpus YORÙLECT across three domains and four regional Yorùbá dialects. To develop this corpus, we engaged native speakers, travelling to communities where these dialects are spoken, to collect text and speech data. Using our newly created corpus, we conducted extensive experiments on (text) machine translation, automatic speech recognition, and speech-to-text translation. Our results reveal substantial performance disparities between standard Yorùbá and the other dialects across all tasks. However, we also show that with dialect-adaptive finetuning, we are able to narrow this gap. We believe our dataset and experimental analysis will contribute greatly to developing NLP tools for Yorùbá and its dialects, and potentially for other African languages, by improving our understanding of existing challenges and offering a high-quality dataset for further development. We release YORÙLECT dataset and models publicly under an open license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
