推进多语言预训练：面向多语言语言模型的三语文档级预训练 TRIP
Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models
AI总结：
针对多语言预训练依赖单语和双语语料而限制文档级跨语言任务性能的问题，提出TRIP方法，利用三语平行语料并通过Grafting方法将预训练目标加速为三语目标，在多个基准上取得SOTA结果。
摘要：
尽管多语言序列到序列预训练取得了成功，但大多数现有方法依赖于多种语言中的文档级单语语料库、句子级双语语料库（本文中，我们使用“双语语料库”指代包含多种语言对中“双语翻译对”的平行语料库，每个翻译对由两种不同语言中具有相同含义的两个句子/文档组成。我们使用“三语语料库”指代包含多种语言组合中“三语翻译对”的平行语料库，每个翻译对由三个句子/文档组成），以及有时合成的文档级双语语料库。这阻碍了跨语言文档级任务（如文档级翻译）的性能。因此，我们提出挖掘并利用文档级三语平行语料库来改进序列到序列多语言预训练。我们提出了三角文档级预训练（TRIP），这是该领域首个将传统的单语和双语目标加速为三语目标的方法，采用一种名为Grafting的新方法。实验表明，TRIP在三个多语言文档级机器翻译基准和一个跨语言抽象式摘要基准上取得了多项强大的最先进（SOTA）分数，包括最高提升3.11个d-BLEU点和8.9个ROUGE-L点的一致改进。
英文摘要：
Despite the success of multilingual sequence-to-sequence pre-training, most existing approaches rely on document-level monolingual corpora in many different languages, sentence-level bilingual corpora,\footnote{In this paper, we use `bilingual corpora' to denote parallel corpora with `bilingual translation pairs' in many different language pairs, each consisting of two sentences/documents with the same meaning written in different languages. We use `trilingual corpora' to denote parallel corpora with `trilingual translation pairs' in many different language combinations, each consisting of three sentences/documents.} and sometimes synthetic document-level bilingual corpora. This hampers the performance with cross-lingual document-level tasks such as document-level translation. Therefore, we propose to mine and leverage document-level trilingual parallel corpora to improve sequence-to-sequence multilingual pre-training. We present \textbf{Tri}angular Document-level \textbf{P}re-training (\textbf{TRIP}), which is the first in the field to accelerate the conventional monolingual and bilingual objectives into a trilingual objective with a novel method called Grafting. Experiments show that TRIP achieves several strong state-of-the-art (SOTA) scores on three multilingual document-level machine translation benchmarks and one cross-lingual abstractive summarization benchmark, including consistent improvements by up to 3.11 d-BLEU points and 8.9 ROUGE-L points.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
