从单语到多语：评估Mamba在南非语言中的ASR表现
From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
AI总结：
本研究评估Mamba模型在南非七种语言上的ASR性能，发现其与Conformer精度相当但计算更高效，并探索多语训练及语言嵌入的影响。
摘要：
自动语音识别（ASR）的最新进展探索了不同的序列模型，包括基于Conformer的模型和更新的状态空间模型（如Mamba）。尽管先前的工作已在多种语言中评估了这些架构，但它们在非洲语言中的有效性仍未得到充分探索。在这项工作中，我们评估了Mamba在七种南非语言上的ASR性能。在单语实验中，每个模型在每种语言的50小时语音上训练，我们将Mamba与类似参数规模的Conformer基线进行比较。Mamba在识别精度上与Conformer相当，同时使用更少的计算资源且训练更快。我们进一步评估了该设置下的泛化能力，发现两个模型都难以泛化到比训练数据长得多的语音。然后，我们使用Mamba模型研究多语ASR，基线是将所有语言合并在一起。在此基础上，我们测试了三种扩展：通过将语言和语系嵌入作为偏置添加到下采样的声学表示中，利用语系信息进行训练；以及使用CTC ASR目标和语言识别（LID）头的多任务学习。我们发现多语训练持续优于单语训练。然而，添加显式语言信息并不会提高域内性能，但会提高跨语料库的鲁棒性。我们在低资源多语设置中进行了消融研究，使用每种语言5小时和10小时的训练数据，观察到使用语言嵌入带来的收益，并进一步证明移除或改变它们会损害模型性能。最后，我们分析了这些嵌入，发现它们并未在类型学意义上捕捉语言相似性，而是充当特定任务的控制向量。
英文摘要：
Recent advances in automatic speech recognition (ASR) have explored different sequence models, including Conformer-based models and newer state space models such as Mamba. Although prior work has evaluated these architectures in multiple languages, their effectiveness in African languages remains underexplored. In this work, we evaluate Mamba for ASR on seven South African languages. In monolingual experiments, each model is trained on 50 hours of speech per language, and we compare Mamba to a Conformer baseline of similar parameter scale. Mamba achieves similar recognition accuracy to Conformer while using fewer computational resources and training faster. We further evaluate generalization in this setting and find that both models struggle to generalize to speech that is much longer than what they were trained on. We then study multilingual ASR using Mamba models, where the baseline is pooling all languages together. On top of this, we tested three extensions: training with language-family information by adding both language and language-family embeddings as biases to the downsampled acoustic representations, and multitask learning with a CTC ASR objective and a language identification (LID) head. We find that multilingual training consistently improves performance over monolingual training. However, adding explicit language information does not improve in-domain performance but does improve cross-corpus robustness. We conducted ablation studies in low-resource multilingual settings using 5-hour and 10-hour per-language training data, where we observed gains from using language embeddings and further demonstrated that removing or altering them hurt model performance. Lastly, we analysed these embeddings and find that they do not capture linguistic similarity in a typological sense, but instead act as task-specific control vectors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
