窗口化摘要混合：一种高效的自监督学习模型微调方法用于低资源语音识别
Windowed SummaryMixing: An Efficient Fine-Tuning of Self-Supervised Learning Models for Low-resource Speech Recognition
AI总结：
本文提出窗口化摘要混合方法，通过整合局部与全局摘要提升自监督学习模型的效率和性能，适用于低资源语音识别场景。
摘要：
自监督学习（SSL）在语音处理方面有所进展，但因自注意力机制导致二次复杂度。为了解决这个问题，摘要混合（SM）被提出作为线性时间的替代方案，通过均值池化来总结整个语音，但缺乏足够的局部上下文。在本工作中，我们引入了窗口化摘要混合（WSM），通过整合局部邻域摘要和全局摘要来增强SM，保持了效率的同时提高了时间依赖性。此外，我们还提出了一种选择性微调方法，用WSM块替代SSL模型中的自注意力层，并在低资源环境下仅微调这些块。我们的方法在提高ASR性能的同时，将SSL模型的峰值VRAM使用量减少了40%。WSM块具有线性时间复杂度和增强的上下文意识。选择性替换一些注意力层减少了计算、内存和延迟，使其成为低资源语音识别的理想选择。
英文摘要：
Self-supervised learning (SSL) has advanced speech processing but suffers from quadratic complexity due to self-attention. To address this, SummaryMixing (SM) has been proposed as a linear-time alternative that summarizes entire utterances using mean pooling but lacks sufficient local context. In this work, we introduce Windowed SummaryMixing (WSM), which enhances SM by integrating local neighborhood summaries alongside the global summary, maintaining efficiency while improving temporal dependencies. Additionally, we introduce a selective fine-tuning approach, replacing self-attention layers in SSL models with WSM blocks and fine-tuning only these blocks in low-resource settings. Our approach improves ASR performance while reducing peak VRAM usage by 40\% in the SSL models. WSM blocks have linear-time complexity with enhanced context awareness. Selectively replacing some attention layers reduces compute, memory, and latency, making it ideal for low-resource speech recognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
