跨语言低资源自动语音识别的顺序适配器堆叠
Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR
- University of Cambridge（剑桥大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对Whisper跨语言低资源ASR，提出顺序适配器堆叠方法，在冻结源适配器上训练目标适配器，在三种语言上较全微调显著降低WER 5-8%。
AI中文摘要：
将大规模多语言自动语音识别（ASR）模型扩展到低资源语言仍然具有挑战性。模型性能偏向于高资源语言，而对于标注数据有限和预训练暴露较少的语言，性能急剧下降。为了解决这个问题，我们研究了在Whisper上将知识从资源丰富的源语言迁移到低资源目标语言的参数高效方法。除了热初始化和基于注意力的融合之外，我们提出了顺序适配器堆叠，该方法在冻结的源语言适配器之上放置一个可训练的目标语言适配器。在受控实验中，这些方法在Whisper不支持的三种目标语言——阿斯图里亚斯语、阿萨姆语和科萨语——上进行了评估，使用了相关程度不同的源语言。使用最接近相关源语言的顺序适配器堆叠在三个目标上始终显著优于完全微调，相对词错误率（WER）降低5%至8%。这些收益在仅有一小时目标训练数据的情况下基本保持。
英文摘要：
Extending large-scale multilingual automatic speech recognition (ASR) models to low-resource languages remains challenging. Model performance is skewed toward high-resource languages and degrades sharply for languages with limited labeled data and pre-training exposure. To address this, we investigate parameter-efficient approaches for transferring knowledge from resource-rich source languages to low-resource target languages on Whisper. Alongside warm initialization and attention-based fusion, we propose Sequential Adapter Stacking, which places a trainable target-language adapter on top of a frozen source-language adapter. Under controlled experiments, these approaches are evaluated on three target languages unsupported by Whisper -- Asturian, Assamese, and Xhosa -- using source languages with varying degrees of relatedness. Sequential Adapter Stacking with the closest related source consistently and significantly outperforms full fine-tuning across the three targets, with 5--8\% relative WER reductions. These gains largely persist with only one hour of target training data.
