非洲语言自然语言推理评估的样本量缩放
Sample-Size Scaling of the African Languages NLI Evaluation
AI总结：
本研究通过AfriXNLI基准对16种非洲语言进行系统样本量缩放实验，发现NLI性能随样本量增加并非单调提升，而是呈现语言敏感且非单调的缩放行为，表明数据量不足以保证稳定收益，需语言敏感的数据集和更强多语言建模策略。
摘要：
非洲语言标注数据非常少，且增加标注数据量是否能可靠提升下游性能尚不明确。本研究基于AfriXNLI基准，对16种非洲语言进行了自然语言推理（NLI）的系统样本量缩放研究。在受控条件下，测试了两个约0.6B参数的多语言Transformer模型（在XNLI上微调的XLM-R Large和AfroXLM-R Large），样本量从50到500个标注示例不等，并在随机子采样运行中平均结果。与通常认为的随数据增加性能单调提升相反，我们发现了一种强烈语言敏感且通常非单调的缩放行为。一些语言在低资源场景下表现出早期饱和或性能下降，以及高方差。这些结果表明，数据量不足以保证非洲NLI的稳定收益，因此需要创建语言敏感的数据集和更强的多语言建模策略。
英文摘要：
African languages have very little labelled data, and it is unclear if augmenting the quantity of annotation data reliably enhances downstream performance. The study is a systematic sample-size scaling study of natural language inference (NLI) on 16 African languages based on the AfriXNLI benchmark. Under controlled conditions, two multilingual transformer models with roughly 0.6B parameters XLM-R Large fine-tuned on XNLI and AfroXLM-R Large are tested on sample sizes of between 50 and 500 labeled examples and average their results across random subsampling runs. As opposed to the usual belief of monotonic increase with increased data, we find a strongly language sensitive and often non-monotonic scaling behavior. Some languages show early saturation or decrease in performance with sample size as well as high variance in low resource regimes. These results indicate that the volume of data is not enough to guarantee stable profits to African NLI, creating the necessity of language sensitive datasets creation and stronger multi-lingual modelling strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
