利用来自斯瓦希里语的迁移学习：推进科摩罗方言的解决方案
Harnessing Transfer Learning from Swahili: Advancing Solutions for Comorian Dialects
AI总结：
本研究利用斯瓦希里语到科摩罗语的迁移学习，通过计算词汇距离构建混合数据集，在ASR和MT任务上实证验证了跨语言理解假设，为低资源科摩罗语开创了NLP技术并促进其语言遗产保护。
摘要：
如果今天一些非洲语言（如斯瓦希里语）拥有足够的资源来开发高性能的自然语言处理（NLP）系统，但该大陆上使用的许多其他语言仍然缺乏这种支持。对于这些仍处于起步阶段的语言，存在几种可能性来解决这种严重的数据匮乏。其中之一是迁移学习，它允许低资源语言从与它们相似的其他语言的良好表示中受益。在这项工作中，我们采用了类似的方法，旨在为科摩罗语开创NLP技术，科摩罗语是属于班图语系的四种语言或方言的统称。我们的方法最初受到一个假设的启发：如果人类可以毫不费力或只需很少努力就能理解不同于其母语的语言，那么在机器上对这一过程进行建模将完全是可能的。为了实现这一目标，我们考虑了构建混合了斯瓦希里语的科摩罗语数据集的方法。这里需要注意的一点是，在斯瓦希里语数据方面，我们仅通过计算候选数据与源数据之间的词汇距离来关注与科摩罗语最接近的元素。我们在两个用例中实证测试了这一假设：自动语音识别（ASR）和机器翻译（MT）。我们的MT模型分别获得了0.6826、0.42和0.6532的ROUGE-1、ROUGE-2和ROUGE-L分数，而我们的ASR系统记录了39.50%的WER和13.76%的CER。这项研究对于推进代表性不足语言的NLP至关重要，具有在数字时代保护和促进科摩罗语言遗产的潜力。
英文摘要：
If today some African languages like Swahili have enough resources to develop high-performing Natural Language Processing (NLP) systems, many other languages spoken on the continent are still lacking such support. For these languages, still in their infancy, several possibilities exist to address this critical lack of data. Among them is Transfer Learning, which allows low-resource languages to benefit from the good representation of other languages that are similar to them. In this work, we adopt a similar approach, aiming to pioneer NLP technologies for Comorian, a group of four languages or dialects belonging to the Bantu family. Our approach is initially motivated by the hypothesis that if a human can understand a different language from their native language with little or no effort, it would be entirely possible to model this process on a machine. To achieve this, we consider ways to construct Comorian datasets mixed with Swahili. One thing to note here is that in terms of Swahili data, we only focus on elements that are closest to Comorian by calculating lexical distances between candidate and source data. We empirically test this hypothesis in two use cases: Automatic Speech Recognition (ASR) and Machine Translation (MT). Our MT model achieved ROUGE-1, ROUGE-2, and ROUGE-L scores of 0.6826, 0.42, and 0.6532, respectively, while our ASR system recorded a WER of 39.50\% and a CER of 13.76\%. This research is crucial for advancing NLP in underrepresented languages, with potential to preserve and promote Comorian linguistic heritage in the digital age.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
