ChatGPT 超越英语：面向多语言学习中大语言模型的全面评估
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning
AI总结：
本文在 7 个任务、37 种不同资源水平语言上系统评估 ChatGPT 的零样本多语言能力，发现其整体表现不及先前模型，呼吁加强多语言学习研究。
摘要：
在过去几年中，大语言模型（LLMs）已成为自然语言处理（NLP）领域最重要的突破，从根本上改变了该领域的研究与发展。ChatGPT 是近期开发的最令人兴奋的 LLM 系统之一，展示了令人印象深刻的语言生成能力，并高度吸引了公众关注。在英语中发现的 ChatGPT 各种令人兴奋的应用中，由于其多语言训练数据，该模型能够处理并生成多种语言的文本。鉴于 ChatGPT 在英语中已被广泛用于不同问题和领域，一个自然的问题是：ChatGPT 是否也能有效应用于其他语言，还是有必要开发更多针对特定语言的技术。回答这一问题需要对 ChatGPT 在多种语言和大型数据集上的多个任务进行彻底评估（即超越已报道的轶事），而当前研究中此类评估仍然缺失或有限。我们的工作旨在填补 ChatGPT 及类似 LLM 评估方面的这一空白，为多语言 NLP 应用提供更全面的信息。虽然这项工作将持续推进并在未来纳入更多实验，但我们当前的论文在 7 个不同任务上评估了 ChatGPT，涵盖 37 种具有高、中、低和极低资源水平的多样语言。我们还聚焦于 ChatGPT 的零样本学习设置，以提高可复现性并更好地模拟普通用户的交互。与先前模型的性能相比，我们广泛的实验结果表明，ChatGPT 在不同 NLP 任务和语言上的性能较差，这呼吁进一步研究以开发更好的模型并加深对多语言学习的理解。
英文摘要：
Over the last few years, large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) that fundamentally transform research and developments in the field. ChatGPT represents one of the most exciting LLM systems developed recently to showcase impressive skills for language generation and highly attract public attention. Among various exciting applications discovered for ChatGPT in English, the model can process and generate texts for multiple languages due to its multilingual training data. Given the broad adoption of ChatGPT for English in different problems and areas, a natural question is whether ChatGPT can also be applied effectively for other languages or it is necessary to develop more language-specific technologies. The answer to this question requires a thorough evaluation of ChatGPT over multiple tasks with diverse languages and large datasets (i.e., beyond reported anecdotes), which is still missing or limited in current research. Our work aims to fill this gap for the evaluation of ChatGPT and similar LLMs to provide more comprehensive information for multilingual NLP applications. While this work will be an ongoing effort to include additional experiments in the future, our current paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. We also focus on the zero-shot learning setting for ChatGPT to improve reproducibility and better simulate the interactions of general users. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages, calling for further research to develop better models and understanding for multilingual learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
