用英语提问更好：大型语言模型在英语、低资源语言与跨语言场景下的评估
Better to Ask in English: Evaluation of Large Language Models on English, Low-resource and Cross-Lingual Settings
AI总结：
本研究通过零样本提示和五种提示设置评估GPT-4、Llama 2和Gemini在英语与南亚低资源语言中的表现，发现GPT-4全面领先且所有模型在英语提示下效果更好，揭示了低资源语言场景下LLMs的改进需求。
摘要：
大型语言模型（LLMs）在海量数据上训练，使其能够应用于多样化的领域和任务。尽管它们表现出色，但大多数LLMs主要用英语开发和评估。最近出现了一些多语言LLMs，但它们在低资源语言中的表现，尤其是南亚使用人数最多的语言，尚未得到充分探索。为弥补这一空白，本研究评估了GPT-4、Llama 2和Gemini等LLMs，分析它们在英语与南亚其他低资源语言（如孟加拉语、印地语和乌尔都语）中的有效性。具体而言，我们采用零样本提示和五种不同的提示设置，广泛考察LLMs在跨语言翻译提示中的有效性。研究结果表明，GPT-4在所有五种提示设置和所有语言中均优于Llama 2和Gemini。此外，三个LLMs在英语提示下的表现均优于其他低资源语言提示。本研究广泛考察了LLMs在低资源语言语境中的表现，以突出LLMs和语言特定资源所需的改进，从而开发更具通用目的的NLP应用。
英文摘要：
Large Language Models (LLMs) are trained on massive amounts of data, enabling their application across diverse domains and tasks. Despite their remarkable performance, most LLMs are developed and evaluated primarily in English. Recently, a few multi-lingual LLMs have emerged, but their performance in low-resource languages, especially the most spoken languages in South Asia, is less explored. To address this gap, in this study, we evaluate LLMs such as GPT-4, Llama 2, and Gemini to analyze their effectiveness in English compared to other low-resource languages from South Asia (e.g., Bangla, Hindi, and Urdu). Specifically, we utilized zero-shot prompting and five different prompt settings to extensively investigate the effectiveness of the LLMs in cross-lingual translated prompts. The findings of the study suggest that GPT-4 outperformed Llama 2 and Gemini in all five prompt settings and across all languages. Moreover, all three LLMs performed better for English language prompts than other low-resource language prompts. This study extensively investigates LLMs in low-resource language contexts to highlight the improvements required in LLMs and language-specific resources to develop more generally purposed NLP applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
