损失可成福祉：将自监督语音表示路由至高效多语言与多任务语音处理
Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing
AI总结：
本文提出 S$^3$-Router 框架，通过仅微调连接并丢弃不超过 10% 的语音自监督模型权重，在提升效率的同时缓解过拟合，实现多语言多任务语音处理、ASR 剪枝与表示分析的一体化方案。
摘要：
针对丰富语音表示的自监督学习（SSL）已在低资源自动语音识别（ASR）及其他语音处理任务中取得实证成功，这可以减轻对大量转写语音的需求，从而推动了对端侧 ASR 和其他语音处理日益增长的需求。然而，先进的语音 SSL 模型已变得越来越大，这与有限的端侧资源相矛盾。在需要同时识别多种语言或执行多个语音处理任务的多语言/多任务场景中，这一差距可能更加严重。此外，严重过参数化的语音 SSL 模型在低资源语音语料库上进行微调时往往容易出现过拟合。本工作旨在通过我们提出的 S$^3$-Router 框架，提升语音 SSL 模型的实际可用性，在提高效率和缓解过拟合两方面实现双赢。该框架首次发现，仅通过微调语音 SSL 模型的模型连接，丢弃不超过 10% 的模型权重，就能在下游语音处理任务上取得优于标准权重微调的准确率。更重要的是，S$^3$-Router 可以作为一种一体化技术，实现：（1）一种新的微调方案，（2）一种高效的多语言/多任务解决方案，（3）一种最先进的 ASR 剪枝技术，以及（4）一种定量分析所学语音表示的新工具。我们相信 S$^3$-Router 为语音 SSL 模型的实际部署提供了新的视角。我们的代码可在以下地址获取：https://github.com/GATECH-EIC/S3-Router。
英文摘要：
Self-supervised learning (SSL) for rich speech representations has achieved empirical success in low-resource Automatic Speech Recognition (ASR) and other speech processing tasks, which can mitigate the necessity of a large amount of transcribed speech and thus has driven a growing demand for on-device ASR and other speech processing. However, advanced speech SSL models have become increasingly large, which contradicts the limited on-device resources. This gap could be more severe in multilingual/multitask scenarios requiring simultaneously recognizing multiple languages or executing multiple speech processing tasks. Additionally, strongly overparameterized speech SSL models tend to suffer from overfitting when being finetuned on low-resource speech corpus. This work aims to enhance the practical usage of speech SSL models towards a win-win in both enhanced efficiency and alleviated overfitting via our proposed S$^3$-Router framework, which for the first time discovers that simply discarding no more than 10\% of model weights via only finetuning model connections of speech SSL models can achieve better accuracy over standard weight finetuning on downstream speech processing tasks. More importantly, S$^3$-Router can serve as an all-in-one technique to enable (1) a new finetuning scheme, (2) an efficient multilingual/multitask solution, (3) a state-of-the-art ASR pruning technique, and (4) a new tool to quantitatively analyze the learned speech representation. We believe S$^3$-Router has provided a new perspective for practical deployment of speech SSL models. Our codes are available at: https://github.com/GATECH-EIC/S3-Router.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
