P-MMEval：用于一致评估 LLM 的并行多语言多任务基准
P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs
AI总结：
本文提出并行多语言多任务基准 P-MMEval，通过一致语言覆盖与并行样本评估 LLM，并基于多模型实验分析任务、规模、语言和提示词等因素及英语知识迁移效果。
摘要：
近期，大型语言模型（LLM）的进展展现出其在翻译、代码生成和推理等任务上各不相同的多语言能力。以往评估往往将范围局限于基础自然语言处理（NLP）或彼此孤立、针对特定能力的任务。为缓解这一缺陷，我们旨在提出一个全面的多语言多任务基准。首先，我们推出 P-MMEval，这是一个覆盖有效基础数据集与能力专用数据集的大规模基准。此外，P-MMEval 在各类数据集之间提供一致的语言覆盖，并提供并行样本。最后，我们在具有代表性的多语言模型系列上开展大量实验，以比较不同模型和任务之间的性能，探究多语言性能与任务、模型规模、语言和提示词等因素之间的关系，并检验从英语到其他语言的知识迁移效果。所得洞见旨在为未来研究提供有价值的指导。该数据集可在 https://huggingface.co/datasets/Qwen/P-MMEval 获取。
英文摘要：
Recent advancements in large language models (LLMs) showcase varied multilingual capabilities across tasks like translation, code generation, and reasoning. Previous assessments often limited their scope to fundamental natural language processing (NLP) or isolated capability-specific tasks. To alleviate this drawback, we aim to present a comprehensive multilingual multitask benchmark. First, we introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets. Furthermore, P-MMEval delivers consistent language coverage across various datasets and provides parallel samples. Finally, we conduct extensive experiments on representative multilingual model series to compare performances across models and tasks, explore the relationship between multilingual performances and factors such as tasks, model sizes, languages, and prompts, and examine the effectiveness of knowledge transfer from English to other languages. The resulting insights are intended to offer valuable guidance for future research. The dataset is available at https://huggingface.co/datasets/Qwen/P-MMEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
