面向低资源多口音自动语音识别的LoRA专家混合模型
Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition
AI总结：
针对低资源多口音非母语语音识别鲁棒性不足的问题，提出MAS-LoRA微调方法，通过混合不同口音的LoRA专家实现推理时口音已知/未知场景下的适配，在L2-ARCTIC语料库上基于Whisper取得优于常规LoRA和全微调的词错误率，且灾难性遗忘更少。
摘要：
我们旨在提升自动语音识别（ASR）系统对非母语语音的鲁棒性，尤其是在低资源多口音场景下。我们提出了口音特定LoRA混合（MAS-LoRA）方法，这是一种利用低秩适配（LoRA）专家混合的微调方法，每个专家专门适配一种特定口音。该方法可在推理时口音已知或未知的情况下使用，无需重新微调模型。我们在L2-ARCTIC语料库上使用Whisper开展实验，结果表明，当口音未知时，该方法相比常规LoRA和全微调在词错误率上有显著提升；当口音已知时，结果会进一步改善。此外，MAS-LoRA的灾难性遗忘程度低于其他微调方法。据我们所知，这是首次将LoRA专家混合用于非母语多口音ASR研究。
英文摘要：
We aim to improve the robustness of Automatic Speech Recognition (ASR) systems against non-native speech, particularly in low-resourced multi-accent settings. We introduce Mixture of Accent-Specific LoRAs (MAS-LoRA), a fine-tuning method that leverages a mixture of Low-Rank Adaptation (LoRA) experts, each specialized in a specific accent. This method can be used when the accent is known or unknown at inference time, without the need to fine-tune the model again. Our experiments, conducted using Whisper on the L2-ARCTIC corpus, demonstrate significant improvements in Word Error Rate compared to regular LoRA and full fine-tuning when the accent is unknown. When the accent is known, the results further improve. Furthermore, MAS-LoRA shows less catastrophic forgetting than the other fine-tuning methods. To the best of our knowledge, this is the first use of a mixture of LoRA experts for non-native multi-accent ASR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
