面向低资源语言的Transformer模型：我们能做到！
Transformers for Low-Resource Languages: Is Féidir Linn!
AI总结：
针对低资源英语-爱尔兰语翻译任务，研究通过优化Transformer超参数（重点优化子词模型等），实现了较基线RNN模型7.8分的BLEU提升，效果优于Google Translate，验证了Transformer在低资源翻译场景的有效性。
摘要：
Transformer模型是机器翻译领域的当前最优模型。但总体而言，神经翻译模型在训练数据不足的语言对上往往表现不佳，因此针对低资源语言对开展的该架构相关实验相对较少。本研究评估了Transformer模型在低资源英语-爱尔兰语翻译任务中的超参数优化效果。我们证明，选择合适的参数可带来显著的性能提升，其中最重要的是，子词模型的正确选择是翻译性能的最大驱动因素。本研究评估了采用unigram和BPE两种方法的SentencePiece模型；模型架构的调整包括修改层数、测试多种正则化技术，以及评估注意力头的最优数量。实验使用通用的5.5万条DGT语料库和领域内的8.8万条公共行政语料库进行评估。优化后的Transformer模型与基线RNN模型相比，BLEU得分提升了7.8分；包括TER在内的多项指标均有所改善，表明采用1.6万BPE子词模型的优化Transformer模型可大幅降低译后编辑工作量。与Google Translate进行基准对比后，我们的翻译引擎表现出显著提升。本研究解答了Transformer能否在英语-爱尔兰语翻译这一低资源场景中有效应用的问题：Is féidir linn——是的，我们可以。
英文摘要：
The Transformer model is the state-of-the-art in Machine Translation. However, in general, neural translation models often under perform on language pairs with insufficient training data. As a consequence, relatively few experiments have been carried out using this architecture on low-resource language pairs. In this study, hyperparameter optimization of Transformer models in translating the low-resource English-Irish language pair is evaluated. We demonstrate that choosing appropriate parameters leads to considerable performance improvements. Most importantly, the correct choice of subword model is shown to be the biggest driver of translation performance. SentencePiece models using both unigram and BPE approaches were appraised. Variations on model architectures included modifying the number of layers, testing various regularisation techniques and evaluating the optimal number of heads for attention. A generic 55k DGT corpus and an in-domain 88k public admin corpus were used for evaluation. A Transformer optimized model demonstrated a BLEU score improvement of 7.8 points when compared with a baseline RNN model. Improvements were observed across a range of metrics, including TER, indicating a substantially reduced post editing effort for Transformer optimized models with 16k BPE subword models. Bench-marked against Google Translate, our translation engines demonstrated significant improvements. The question of whether or not Transformers can be used effectively in a low-resource setting of English-Irish translation has been addressed. Is féidir linn - yes we can.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
