利用大语言模型驱动的数据增强提升跨语言性能
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance
AI总结：
本文利用大语言模型对多语言常识推理数据集进行数据增强，通过微调较小多语言模型验证其有效性，发现LLM生成数据能显著提升性能，并评估了生成文本的自然度与逻辑一致性。
摘要：
本文探讨了在可用训练数据极其有限的多语言常识推理数据集中，利用大语言模型（LLMs）进行数据增强的潜力。为此，我们使用Dolly-v2、StableVicuna、ChatGPT和GPT-4等LLMs对XCOPA、XWinograd和XStoryCloze三个数据集进行增强。随后，我们评估了使用合成数据微调mBERT和XLMR等较小多语言模型的有效性。我们比较了使用英语和目标语言生成的数据，以及翻译后的英语生成数据进行训练的性能，揭示了引入LLM生成数据的整体优势，例如在最佳情况下准确率提升了13.4。此外，我们邀请母语者对不同语言生成示例的自然度和逻辑连贯性进行人工评估。结果表明，ChatGPT和GPT-4等LLM在大多数语言中能生成自然连贯的文本，但在泰米尔语等某些语言中难以生成有意义的文本。我们还发现，与原始数据集相比，ChatGPT在生成合理替代选项方面存在不足，而GPT-4生成的示例展现出具有竞争力的逻辑一致性。
英文摘要：
This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited. To achieve this, we utilise several LLMs, namely Dolly-v2, StableVicuna, ChatGPT, and GPT-4, to augment three datasets: XCOPA, XWinograd, and XStoryCloze. Subsequently, we evaluate the effectiveness of fine-tuning smaller multilingual models, mBERT and XLMR, using the synthesised data. We compare the performance of training with data generated in English and target languages, as well as translated English-generated data, revealing the overall advantages of incorporating data generated by LLMs, e.g. a notable 13.4 accuracy score improvement for the best case. Furthermore, we conduct a human evaluation by asking native speakers to assess the naturalness and logical coherence of the generated examples across different languages. The results of the evaluation indicate that LLMs such as ChatGPT and GPT-4 excel at producing natural and coherent text in most languages, however, they struggle to generate meaningful text in certain languages like Tamil. We also observe that ChatGPT falls short in generating plausible alternatives compared to the original dataset, whereas examples from GPT-4 exhibit competitive logical consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
