NusaBERT：教 IndoBERT 成为多语言与多文化模型
NusaBERT: Teaching IndoBERT to be Multilingual and Multicultural
AI总结：
本文提出 NusaBERT，通过扩展 IndoBERT 词汇并利用含地区语言、方言的多语言语料继续训练，在印尼多语言任务基准上达到最先进性能。
摘要：
印度尼西亚的语言景观极为多样，涵盖 700 多种语言和方言，使其成为世界上语言资源最丰富的国家之一。这种多样性，加上普遍存在的语码转换现象以及低资源地区语言，给现代预训练语言模型带来了独特挑战。为应对这些挑战，我们开发了 NusaBERT：它以 IndoBERT 为基础，纳入词汇扩展，并利用包含地区语言和方言的多样化多语言语料库。通过在一系列基准上进行严格评估，NusaBERT 在涉及印度尼西亚多种语言的任务中展现出最先进性能，为未来针对代表性不足语言的自然语言理解研究铺平道路。
英文摘要：
Indonesia's linguistic landscape is remarkably diverse, encompassing over 700 languages and dialects, making it one of the world's most linguistically rich nations. This diversity, coupled with the widespread practice of code-switching and the presence of low-resource regional languages, presents unique challenges for modern pre-trained language models. In response to these challenges, we developed NusaBERT, building upon IndoBERT by incorporating vocabulary expansion and leveraging a diverse multilingual corpus that includes regional languages and dialects. Through rigorous evaluation across a range of benchmarks, NusaBERT demonstrates state-of-the-art performance in tasks involving multiple languages of Indonesia, paving the way for future natural language understanding research for under-represented languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
