统一梯度投影：用于多语言低资源语音识别的语言平衡持续学习
Unified Gradient Projection: Language-Balanced Continual Learning for Multilingual Low-Resource ASR
浏览论文内容
中文总结 AI 辅助
针对多语言低资源语音识别中微调导致灾难性遗忘及跨任务干扰问题，提出统一梯度投影（UGP）方法，通过语言平衡重放的参考梯度约束参数更新，结合梯度级与数据级重放，有效减轻遗忘，实现有效适应。
中文摘要 AI 辅助
大规模预训练的语音识别模型如Whisper展现出强大的多语言能力。然而，在低资源语言上微调常导致灾难性遗忘。虽然持续学习可缓解此问题，但现有方法在多语言环境中难以调节跨任务干扰，其中优势语言会影响优化。我们提出统一梯度投影（UGP），它在统一投影空间中使用来自语言平衡重放的参考梯度来约束参数更新。通过均衡投影中每种语言的贡献，UGP减少了优势语言偏差并提高了跨语言稳定性。我们还表明，将梯度级投影与数据级重放相结合可在稳定性和可塑性方面产生互补收益。在不同的低资源语言组和模型规模上，UGP能够有效适应，同时大幅减轻遗忘。在Whisper-large-v3上，它实现了近乎零的平均遗忘。
英文摘要
Large-scale pretrained ASR models such as Whisper exhibit strong multilingual capabilities. However, fine-tuning on low-resource languages often causes catastrophic forgetting. Although continual learning mitigates this issue, existing methods struggle to regulate cross-task interference in multilingual settings, where dominant languages bias optimization. We propose Unified Gradient Projection (UGP), which constrains parameter updates using reference gradients from language-balanced replay in a unified projection space. By equalizing per-language contributions in the projection, UGP reduces dominant-language bias and improves cross-lingual stability. We further show that combining gradient-level projection with data-level replay yields complementary gains in stability and plasticity. Across diverse low-resource language groups and model scales, UGP enables effective adaptation while substantially mitigating forgetting. On Whisper-large-v3, it achieves near-zero average forgetting.
发表机构
- Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
机构由 AI 辅助整理，请以论文原文为准。
