arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.01015cs.CLcs.NEcs.SDeess.AS
收藏

如何估计预训练语音模型的可迁移性?

How to Estimate Model Transferability of Pre-Trained Speech Models?

National Taiwan University(台湾大学);Google(谷歌);Georgia Tech(佐治亚理工学院)

Zih-Ching Chen, Chao-Han Huck Yang, Bo Li, Yu Zhang, Nanxin Chen, Shuo-Yiin Chang, Rohit Prabhavalkar, Hung-yi Lee, Tara N. Sainath

AI总结:

本文提出一种基于评分评估的框架,利用贝叶斯似然估计和最优传输理论,无需实际微调即可高效估计预训练语音模型的可迁移性,实验验证了其与微调结果高度相关且节省计算资源。

摘要:

在本工作中,我们引入了一个“基于评分的评估”框架,用于估计预训练语音模型(PSMs)在微调目标任务时的可迁移性。我们利用两种表示理论,即贝叶斯似然估计和最优传输,来基于提取的表示为PSM候选生成排名分数。我们的框架通过做出时间独立性假设,无需对候选模型或层进行实际微调即可高效计算可迁移性分数。我们使用公开数据在跨层和跨模型设置下评估了一些流行的监督语音模型(如Conformer RNN-Transducer)和自监督语音模型(如HuBERT)。实验结果显示,我们的估计框架与微调真实值之间具有较高的Spearman秩相关系数和较低的p值。我们提出的可迁移性框架需要较少的计算时间和资源,使其成为调整语音基础模型的一种节省资源和时间高效的方法。

英文摘要:

In this work, we introduce a "score-based assessment" framework for estimating the transferability of pre-trained speech models (PSMs) for fine-tuning target tasks. We leverage upon two representation theories, Bayesian likelihood estimation and optimal transport, to generate rank scores for the PSM candidates using the extracted representations. Our framework efficiently computes transferability scores without actual fine-tuning of candidate models or layers by making a temporal independent hypothesis. We evaluate some popular supervised speech models (e.g., Conformer RNN-Transducer) and self-supervised speech models (e.g., HuBERT) in cross-layer and cross-model settings using public data. Experimental results show a high Spearman's rank correlation and low $p$-value between our estimation framework and fine-tuning ground truth. Our proposed transferability framework requires less computational time and resources, making it a resource-saving and time-efficient approach for tuning speech foundation models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑