arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.05740cs.CL
收藏

有效的多语言微调方法的探索--以摘要为例

Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization

Carnegie Mellon University(卡内基梅隆大学);Inspired Cognition

Yiwei Qin, Graham Neubig, Pengfei Liu

摘要:

最近,已经提出了大量的调优策略,以使预训练的语言模型适应下游任务。在本文中,我们进行了广泛的实证评估的各种调整策略的多语言学习,特别是在文本摘要的背景下。具体来说,我们探讨了三个家庭的多语言调整策略(共五种模式)的相对优势,并实证评估他们总结超过45种语言。在实验上,我们不仅在XL-Sum数据集上建立了一个新的最先进的模型,而且还得到了一系列的观察结果,希望这些观察结果可以为未来设计多语言调优策略的研究提供线索。

英文摘要:

Recently, a large number of tuning strategies have been proposed to adapt pre-trained language models to downstream tasks. In this paper, we perform an extensive empirical evaluation of various tuning strategies for multilingual learning, particularly in the context of text summarization. Specifically, we explore the relative advantages of three families of multilingual tuning strategies (a total of five models) and empirically evaluate them for summarization over 45 languages. Experimentally, we not only established a new state-of-the-art on the XL-Sum dataset but also derive a series of observations that hopefully can provide hints for future research on the design of multilingual tuning strategies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑