一种使用 Submodels 的可扩展模型专业化训练与推理框架及其在语音模型个性化中的应用
A Scalable Model Specialization Framework for Training and Inference using Submodels and its Application to Speech Model Personalization
AI总结:
提出一种基于 Submodels 的模块化框架,通过动态加载和并行训练提升模型训练与推理的可扩展性,在语音个性化中实现128倍吞吐量提升且不损失精度。
摘要:
模型微调与自适应已成为针对下游任务或领域进行模型专业化的常用方法。对整个模型或部分参数使用轻量级自适应进行微调,在不同的专业化任务中取得了显著成功。为大量领域微调模型通常需要为每个领域启动新的训练任务,从而带来扩展性限制。这些模型训练完成后,在实时应用中部署它们也对推理提出了重大的可扩展性挑战。本文在先前的轻量级自适应技术基础上,提出了一个模块化框架,能够大幅提升模型训练和推理的可扩展性。我们引入了 Submodels,可以快速动态加载以实现即时推理。我们还提出了多种方法,利用同一训练任务中的嵌入空间并行训练这些 Submodels。我们在一个极端用例上测试了该框架,即针对非典型语音的语音模型个性化,要求为每个用户提供一个 Submodel。在固定计算预算下,我们在不损失精度的前提下获得了 128 倍的 Submodel 吞吐量。我们还表明,学习说话人嵌入空间可以进一步扩展,并减少每个说话人所需的个性化训练数据量。
英文摘要:
Model fine-tuning and adaptation have become a common approach for model specialization for downstream tasks or domains. Fine-tuning the entire model or a subset of the parameters using light-weight adaptation has shown considerable success across different specialization tasks. Fine-tuning a model for a large number of domains typically requires starting a new training job for every domain posing scaling limitations. Once these models are trained, deploying them also poses significant scalability challenges for inference for real-time applications. In this paper, building upon prior light-weight adaptation techniques, we propose a modular framework that enables us to substantially improve scalability for model training and inference. We introduce Submodels that can be quickly and dynamically loaded for on-the-fly inference. We also propose multiple approaches for training those Submodels in parallel using an embedding space in the same training job. We test our framework on an extreme use-case which is speech model personalization for atypical speech, requiring a Submodel for each user. We obtain 128x Submodel throughput with a fixed computation budget without a loss of accuracy. We also show that learning a speaker-embedding space can scale further and reduce the amount of personalization training data required per speaker.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。