arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.06996cs.CLcs.AI
收藏

XNLIeu:一个面向巴斯克语的跨语言自然语言推理数据集

XNLIeu: a dataset for cross-lingual NLI in Basque

HiTZ Center - Ixa, University of the Basque Country UPV/EHU(巴斯克大学UPV/EHU HiTZ中心 - Ixa);Orai NLP Technologies(Orai NLP技术公司)

Maite Heredia, Julen Etxaniz, Muitze Zulaika, Xabier Saralegi, Jeremy Barnes, Aitor Soroa

AI总结:

本研究构建了低资源语言巴斯克语的跨语言NLI数据集XNLIeu,经机器翻译加人工后编辑生成,通过实验验证了后编辑的必要性及translate-train策略的整体优势,相关资源已公开。

摘要:

XNLI是一个广泛使用的自然语言推理(NLI)基准,常用于评估跨语言自然语言理解(NLU)能力。本文将XNLI扩展至巴斯克语——这是一种低资源语言,能从迁移学习方法中大幅受益。新数据集名为XNLIeu,其构建流程为先将英文XNLI语料机器翻译为巴斯克语,再进行人工后编辑。我们使用单语和多语大语言模型(LLM)开展了一系列实验,评估三个方面:a)专业后编辑对机器翻译(MT)系统的影响;b)巴斯克语NLI的最优跨语言策略;c)最优跨语言策略的选择是否受数据集由翻译构建这一事实的影响。结果表明,后编辑是必要的,且translate-train跨语言策略整体效果更优,不过在从零开始原生构建的数据集上测试时,该策略的增益会更低。我们的代码和数据集以开放许可公开提供。

英文摘要:

XNLI is a popular Natural Language Inference (NLI) benchmark widely used to evaluate cross-lingual Natural Language Understanding (NLU) capabilities across languages. In this paper, we expand XNLI to include Basque, a low-resource language that can greatly benefit from transfer-learning approaches. The new dataset, dubbed XNLIeu, has been developed by first machine-translating the English XNLI corpus into Basque, followed by a manual post-edition step. We have conducted a series of experiments using mono- and multilingual LLMs to assess a) the effect of professional post-edition on the MT system; b) the best cross-lingual strategy for NLI in Basque; and c) whether the choice of the best cross-lingual strategy is influenced by the fact that the dataset is built by translation. The results show that post-edition is necessary and that the translate-train cross-lingual strategy obtains better results overall, although the gain is lower when tested in a dataset that has been built natively from scratch. Our code and datasets are publicly available under open licenses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑