翻译错误显著影响跨语言学习中的低资源语言
Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning
AI总结：
本研究发现跨语言基准中翻译不一致对低资源语言影响更大，提出通过人工与机器翻译零样本性能差距检测错误，并手动验证了印地语和乌尔都语的翻译问题。
摘要：
用于评估跨语言语言理解的流行基准（如XNLI）由多个目标语言的英文评估集的平行版本组成，这些版本是在专业翻译人员的帮助下创建的。在创建此类平行数据时，确保所有目标语言的高质量翻译对于准确刻画跨语言迁移至关重要。在这项工作中，我们发现翻译不一致确实存在，且有趣的是，它们对XNLI中的低资源语言影响不成比例。为识别此类不一致，我们提出衡量多个目标语言上人工翻译与机器翻译的目标文本的零样本评估性能之间的差距；相对较大的差距表明存在翻译错误。我们还通过手动重新标注印地语和乌尔都语这两种目标语言的人工翻译测试实例，发现这些实例与它们本应继承的原始英文标签一致性较差，从而证实了这两种语言中确实存在翻译错误。
英文摘要：
Popular benchmarks (e.g., XNLI) used to evaluate cross-lingual language understanding consist of parallel versions of English evaluation sets in multiple target languages created with the help of professional translators. When creating such parallel data, it is critical to ensure high-quality translations for all target languages for an accurate characterization of cross-lingual transfer. In this work, we find that translation inconsistencies do exist and interestingly they disproportionally impact low-resource languages in XNLI. To identify such inconsistencies, we propose measuring the gap in performance between zero-shot evaluations on the human-translated and machine-translated target text across multiple target languages; relatively large gaps are indicative of translation errors. We also corroborate that translation errors exist for two target languages, namely Hindi and Urdu, by doing a manual reannotation of human-translated test instances in these two languages and finding poor agreement with the original English labels these instances were supposed to inherit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
