GREEK-BERT：希腊人访问芝麻街
GREEK-BERT: The Greeks visiting Sesame Street
摘要：
基于转换器的语言模型，如BERT及其变体，在通用基准数据自然语言处理（例如，胶水，SQUAD，RACE）。然而，这些模型大多被应用于资源丰富的英语语言。在本文中，我们提出了GREEK-BERT，一个单语BERT为基础的语言模型的现代希腊语。我们在三个NLP任务中评估其性能，即，词性标注、命名实体识别和自然语言推理，获得最先进的性能。有趣的是，在其中两个基准测试中，GREEK-BERT的表现优于两个基于多语言transformer的模型（M-BERT，XLM-R），以及在预训练的词嵌入上操作的较浅神经基线（5%-10%）。最重要的是，我们公开了GREEK-BERT和我们的训练代码，以及说明如何为下游NLP任务微调GREEK-BERT的代码。我们希望这些资源能够促进现代希腊语的NLP研究和应用。
英文摘要：
Transformer-based language models, such as BERT and its variants, have achieved state-of-the-art performance in several downstream natural language processing (NLP) tasks on generic benchmark datasets (e.g., GLUE, SQUAD, RACE). However, these models have mostly been applied to the resource-rich English language. In this paper, we present GREEK-BERT, a monolingual BERT-based language model for modern Greek. We evaluate its performance in three NLP tasks, i.e., part-of-speech tagging, named entity recognition, and natural language inference, obtaining state-of-the-art performance. Interestingly, in two of the benchmarks GREEK-BERT outperforms two multilingual Transformer-based models (M-BERT, XLM-R), as well as shallower neural baselines operating on pre-trained word embeddings, by a large margin (5%-10%). Most importantly, we make both GREEK-BERT and our training code publicly available, along with code illustrating how GREEK-BERT can be fine-tuned for downstream NLP tasks. We expect these resources to boost NLP research and applications for modern Greek.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
