GlotEval：面向大规模多语言大语言模型评估的测试套件
GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models
AI总结：
针对现有评估框架偏重英语和高资源语言的问题，提出轻量级多语言评估框架GlotEval，支持七项任务和数百种语言，实现精准诊断模型在多语言环境中的优劣。
摘要：
大语言模型（LLMs）正在全球以前所未有的速度发展，越来越多的地区采用这些模型来支持其主要语言的应用。在多样化的语言环境中评估这些模型，尤其是在低资源语言中，已成为学术界和工业界面临的主要挑战。现有的评估框架过度集中于英语和少数高资源语言，从而忽视了LLMs在多语言和低资源场景下的真实性能。为弥补这一空白，我们提出了GlotEval，一个专为大规模多语言评估设计的轻量级框架。该框架支持七项关键任务（机器翻译、文本分类、摘要生成、开放式生成、阅读理解、序列标注和内在评估），覆盖数十至数百种语言，并强调一致的多语言基准测试、语言特定的提示模板以及非英语中心的机器翻译。这使得能够在多样化的语言环境中精确诊断模型的优势与不足。一项多语言翻译案例研究展示了GlotEval在多语言及语言特定评估中的适用性。
英文摘要：
Large language models (LLMs) are advancing at an unprecedented pace globally, with regions increasingly adopting these models for applications in their primary language. Evaluation of these models in diverse linguistic environments, especially in low-resource languages, has become a major challenge for academia and industry. Existing evaluation frameworks are disproportionately focused on English and a handful of high-resource languages, thereby overlooking the realistic performance of LLMs in multilingual and lower-resource scenarios. To address this gap, we introduce GlotEval, a lightweight framework designed for massively multilingual evaluation. Supporting seven key tasks (machine translation, text classification, summarization, open-ended generation, reading comprehension, sequence labeling, and intrinsic evaluation), spanning over dozens to hundreds of languages, GlotEval highlights consistent multilingual benchmarking, language-specific prompt templates, and non-English-centric machine translation. This enables a precise diagnosis of model strengths and weaknesses in diverse linguistic contexts. A multilingual translation case study demonstrates GlotEval's applicability for multilingual and language-specific evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
