VLUE：面向越南语自然语言理解的新基准与多任务知识迁移学习
VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding
AI总结：
本文提出了首个越南语自然语言理解评估基准 VLUE，涵盖五个数据集，并提出基于 XLM-RoBERTa 的新模型 CafeBERT，通过额外越南语预训练在所有任务上取得最优结果。
摘要：
各种语言的自然语言理解（NLU）基准（如英语的 GLUE、中文的 CLUE、韩语的 KLUE 和印尼语的 IndoNLU）的成功，促进了在各种任务上对新 NLU 模型的评估。为了建立越南语 NLU 的标准化基准集，我们引入了首个越南语语言理解评估（VLUE）基准。VLUE 基准包含涵盖不同 NLU 任务（包括文本分类、片段抽取和自然语言理解）的五个数据集。为了提供对越南语 NLU 当前状态的深入概述，我们在提出的 VLUE 基准上评估了七个最先进的预训练模型，包括多语言和越南语单语模型。此外，我们提出了 CafeBERT，这是一种新的最先进的预训练模型，在 VLUE 基准的所有任务中都取得了优异的结果。我们的模型结合了多语言预训练模型的能力和越南语语言知识。CafeBERT 基于 XLM-RoBERTa 模型开发，通过利用大量越南语文本数据进行额外的预训练步骤，增强了其对越南语的适应性。为了未来研究的目的，CafeBERT 已公开提供用于研究目的。
英文摘要：
The success of Natural Language Understanding (NLU) benchmarks in various languages, such as GLUE for English, CLUE for Chinese, KLUE for Korean, and IndoNLU for Indonesian, has facilitated the evaluation of new NLU models across a wide range of tasks. To establish a standardized set of benchmarks for Vietnamese NLU, we introduce the first Vietnamese Language Understanding Evaluation (VLUE) benchmark. The VLUE benchmark encompasses five datasets covering different NLU tasks, including text classification, span extraction, and natural language understanding. To provide an insightful overview of the current state of Vietnamese NLU, we then evaluate seven state-of-the-art pre-trained models, including both multilingual and Vietnamese monolingual models, on our proposed VLUE benchmark. Furthermore, we present CafeBERT, a new state-of-the-art pre-trained model that achieves superior results across all tasks in the VLUE benchmark. Our model combines the proficiency of a multilingual pre-trained model with Vietnamese linguistic knowledge. CafeBERT is developed based on the XLM-RoBERTa model, with an additional pretraining step utilizing a significant amount of Vietnamese textual data to enhance its adaptation to the Vietnamese language. For the purpose of future research, CafeBERT is made publicly available for research purposes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
