提示与适配器调优用于自监督编码器-解码器语音模型
Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Model
AI总结：
本研究针对自监督编码器-解码器语音模型Wav2Seq，提出提示和适配器调优方法，在序列生成任务上显著优于传统微调，并在跨语言和低资源场景下展现优势。
摘要：
提示（prompting）和适配器调优（adapter tuning）已成为微调（fine-tuning, FT）方法的高效替代方案。然而，现有的语音提示研究主要集中于分类任务，在更复杂的序列生成任务上表现不佳。此外，适配器调优主要应用于仅编码器的自监督模型。我们的实验表明，在自监督编码器-解码器模型Wav2Seq上进行提示，在序列生成任务上超越了先前的工作。在语音识别（ASR）中，词错误率实现了显著的53%相对改进，在槽填充任务中F1分数提升了27%。此外，在低资源场景下，提示方法与微调方法性能相当。我们还展示了提示和适配器调优在Wav2Seq上跨语言ASR的可迁移性。当涉及有限的可训练参数时，提示和适配器调优在7种语言上始终优于传统微调。值得注意的是，在低资源场景下，提示方法始终优于适配器调优。
英文摘要：
Prompting and adapter tuning have emerged as efficient alternatives to fine-tuning (FT) methods. However, existing studies on speech prompting focused on classification tasks and failed on more complex sequence generation tasks. Besides, adapter tuning is primarily applied with a focus on encoder-only self-supervised models. Our experiments show that prompting on Wav2Seq, a self-supervised encoder-decoder model, surpasses previous works in sequence generation tasks. It achieves a remarkable 53% relative improvement in word error rate for ASR and a 27% in F1 score for slot filling. Additionally, prompting competes with the FT method in the low-resource scenario. Moreover, we show the transferability of prompting and adapter tuning on Wav2Seq in cross-lingual ASR. When limited trainable parameters are involved, prompting and adapter tuning consistently outperform conventional FT across 7 languages. Notably, in the low-resource scenario, prompting consistently outperforms adapter tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
