Dolphin：一个具有挑战性和多样性的阿拉伯语自然语言生成基准
Dolphin: A Challenging and Diverse Benchmark for Arabic NLG
AI总结：
本文提出了Dolphin基准，包含13项阿拉伯语NLG任务和40个数据集，旨在全面评估阿拉伯语及多语言模型的性能与泛化能力，并提供了公开排行榜和基线。
摘要：
我们提出了Dolphin，这是一个新颖的基准，旨在满足专门针对阿拉伯语各种语言和变体广泛集合的自然语言生成（NLG）评估框架的需求。该基准涵盖了13种不同的NLG任务，包括对话生成、问答、机器翻译、摘要等。Dolphin包含了一个包含40个多样且具有代表性的公共数据集的大型语料库，跨越50个测试划分，经过精心筛选以反映现实场景和阿拉伯语的语言丰富性。它为评估阿拉伯语和多语言模型的性能及泛化能力设定了新标准，有望使研究人员突破现有方法的边界。我们对Dolphin进行了广泛分析，突出了其多样性并指出了当前阿拉伯语NLG研究中的不足。我们还提供了一个交互式且模块化的公开排行榜，并在我们的基准上评估了多个模型，从而设定了强大的基线供研究人员进行比较。
英文摘要：
We present Dolphin, a novel benchmark that addresses the need for a natural language generation (NLG) evaluation framework dedicated to the wide collection of Arabic languages and varieties. The proposed benchmark encompasses a broad range of 13 different NLG tasks, including dialogue generation, question answering, machine translation, summarization, among others. Dolphin comprises a substantial corpus of 40 diverse and representative public datasets across 50 test splits, carefully curated to reflect real-world scenarios and the linguistic richness of Arabic. It sets a new standard for evaluating the performance and generalization capabilities of Arabic and multilingual models, promising to enable researchers to push the boundaries of current methodologies. We provide an extensive analysis of Dolphin, highlighting its diversity and identifying gaps in current Arabic NLG research. We also offer a public leaderboard that is both interactive and modular and evaluate several models on our benchmark, allowing us to set strong baselines against which researchers can compare.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
