arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.11244cs.CLcs.AIcs.LGcs.NEeess.AS
收藏

一种基于预训练通用语音模型的阿拉伯方言识别参数高效学习方法

A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Model

King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Manipal Institute of Technology(马尼帕尔理工学院);Georgia Institute of Technology(佐治亚理工学院);SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence(沙特数据与人工智能局-阿卜杜拉国王科技大学数据科学与人工智能卓越中心);Karolinska Institutet(卡罗林斯卡学院)

Srijith Radhakrishnan, Chao-Han Huck Yang, Sumeer Ahmad Khan, Narsis A. Kiani, David Gomez-Cabrero, Jesper N. Tegner

AI总结:

本研究探索参数高效学习技术改造通用语音模型用于阿拉伯方言识别,设计残差适配器、模型重编程及令牌级标签映射,在ADI-17数据集上以极少参数取得接近微调的效果,降低了训练成本。

摘要:

在本研究中,我们探索参数高效学习(Parameter-Efficient-Learning, PEL)技术,将通用语音(General-Purpose-Speech, GSM)模型改造用于阿拉伯方言识别(Arabic dialect identification, ADI)。具体而言,我们研究了在冻结预训练设置下,将可训练特征融入多层编码器-解码器GSM结构的不同方案。我们的架构包含残差适配器和模型重编程(输入提示)。我们设计了令牌级标签映射,使GSM适配阿拉伯方言识别(ADI)任务。由于众多区域方言在词汇和发音上差异极大,该任务颇具挑战性。我们通过普通微调在ADI-17数据集上取得了新的最优准确率。我们进一步使用PEL方法降低了训练成本,该方法仅使用2.5%的(额外)网络可训练参数,准确率与微调的差距在1.86%以内。我们的研究展示了如何利用开源代码和预训练模型,以小数据集和有限算力实现阿拉伯方言识别。

英文摘要:

In this work, we explore Parameter-Efficient-Learning (PEL) techniques to repurpose a General-Purpose-Speech (GSM) model for Arabic dialect identification (ADI). Specifically, we investigate different setups to incorporate trainable features into a multi-layer encoder-decoder GSM formulation under frozen pre-trained settings. Our architecture includes residual adapter and model reprogramming (input-prompting). We design a token-level label mapping to condition the GSM for Arabic Dialect Identification (ADI). This is challenging due to the high variation in vocabulary and pronunciation among the numerous regional dialects. We achieve new state-of-the-art accuracy on the ADI-17 dataset by vanilla fine-tuning. We further reduce the training budgets with the PEL method, which performs within 1.86% accuracy to fine-tuning using only 2.5% of (extra) network trainable parameters. Our study demonstrates how to identify Arabic dialects using a small dataset and limited computation with open source code and pre-trained models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑