平行宇宙，平行语言：对基于大语言模型的多语言反事实示例生成的全面研究
Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation
AI总结：
本文研究了大语言模型在多语言反事实示例生成中的有效性，发现翻译生成的反事实示例在有效性上优于直接生成，但质量仍不如英文反事实示例，并揭示了多语言反事实数据增强对模型性能的提升效果。
摘要：
反事实指的是经过最小修改的输入，使其模型预测发生变化，是解释模型行为的一种有前景的方法。大型语言模型（LLMs）在生成英文反事实方面表现出色，并展示了多语言能力。然而，它们在生成多语言反事实方面的有效性仍不明确。为此，我们对多语言反事实进行了全面研究。我们首先对六种语言中直接生成的反事实和通过英文翻译派生的反事实进行了自动评估。尽管基于翻译的反事实比直接生成的反事实更具有效性，但它们需要显著更多的修改，仍无法匹配原始英文反事实的质量。其次，我们发现对高资源欧洲语言反事实应用的编辑模式非常相似，表明跨语言扰动遵循共同的战略原则。第三，我们识别并分类了四类在各语言中持续出现的错误。最后，我们发现多语言反事实数据增强（CDA）比跨语言CDA对模型性能的提升更大，尤其是对低资源语言。然而，生成的反事实的不完善限制了模型性能和鲁棒性的提升。
英文摘要：
Counterfactuals refer to minimally edited inputs that cause a model's prediction to change, serving as a promising approach to explaining the model's behavior. Large language models (LLMs) excel at generating English counterfactuals and demonstrate multilingual proficiency. However, their effectiveness in generating multilingual counterfactuals remains unclear. To this end, we conduct a comprehensive study on multilingual counterfactuals. We first conduct automatic evaluations on both directly generated counterfactuals in the target languages and those derived via English translation across six languages. Although translation-based counterfactuals offer higher validity than their directly generated counterparts, they demand substantially more modifications and still fall short of matching the quality of the original English counterfactuals. Second, we find the patterns of edits applied to high-resource European-language counterfactuals to be remarkably similar, suggesting that cross-lingual perturbations follow common strategic principles. Third, we identify and categorize four main types of errors that consistently appear in the generated counterfactuals across languages. Finally, we reveal that multilingual counterfactual data augmentation (CDA) yields larger model performance improvements than cross-lingual CDA, especially for lower-resource languages. Yet, the imperfections of the generated counterfactuals limit gains in model performance and robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
