arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.04212cs.CL
收藏

解锁低资源语言翻译的参数高效微调

Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation

University of Toronto(多伦多大学);Saarland University(萨尔大学);Massey University(梅西大学);Ontario Tech University(安大略理工大学)

Tong Su, Xin Peng, Sarubi Thillainathan, David Guzmán, Surangika Ranathunga, En-Shiun Annie Lee

AI总结:

本文通过多领域、多规模的实验评估了15种PEFT架构在低资源语言神经机器翻译中的表现,发现6种架构优于基线,其中Houlsby+Inversion适配器效果最佳,验证了PEFT方法的有效性。

摘要:

参数高效微调(PEFT)方法在将大规模预训练语言模型适配到多样任务中日益重要,它在适应性与计算效率之间提供了平衡。这些方法在低资源语言(LRL)神经机器翻译(NMT)中尤为重要,能够在最少资源条件下提升翻译准确性。然而,它们在不同语言上的实际效果差异显著。我们开展了全面的实证实验,涵盖不同的低资源语言领域和规模,使用SacreBLEU分数评估了8种PEFT方法共15种架构的性能。我们表明,6种PEFT架构在域内和域外测试中均优于基线,并且Houlsby+Inversion适配器整体表现最佳,证明了PEFT方法的有效性。

英文摘要:

Parameter-efficient fine-tuning (PEFT) methods are increasingly vital in adapting large-scale pre-trained language models for diverse tasks, offering a balance between adaptability and computational efficiency. They are important in Low-Resource Language (LRL) Neural Machine Translation (NMT) to enhance translation accuracy with minimal resources. However, their practical effectiveness varies significantly across different languages. We conducted comprehensive empirical experiments with varying LRL domains and sizes to evaluate the performance of 8 PEFT methods with in total of 15 architectures using the SacreBLEU score. We showed that 6 PEFT architectures outperform the baseline for both in-domain and out-domain tests and the Houlsby+Inversion adapter has the best performance overall, proving the effectiveness of PEFT methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑