AfriMTEB 和 AfriE5：为非洲语言构建和适应文本嵌入模型的基准测试
AfriMTEB and AfriE5: Benchmarking and Adapting Text Embedding Models for African Languages
AI总结：
本文提出 AfriMTEB 和 AfriE5，为非洲语言构建和适应文本嵌入模型的基准测试，通过引入新任务和跨语言对比蒸馏方法，提升了模型性能。
摘要：
文本嵌入是许多自然语言处理任务的重要组成部分，例如检索增强生成，这对于防止大型语言模型中的幻觉至关重要。尽管最近发布了大规模多语言 MTEB（MMTEB），但非洲语言仍然代表性不足，现有任务通常是从翻译基准如 FLORES 分类或 SIB-200 重新利用的。在本文中，我们介绍了 AfriMTEB——MMTEB 的区域扩展，涵盖 59 种语言、14 个任务和 38 个数据集，包括六个新增的数据集。与许多 MMTEB 数据集包含少于五种语言不同，新增的数据集覆盖了 14 到 56 种非洲语言，并引入了全新的任务，如仇恨言论检测、意图检测和情感分类，这些任务此前未被覆盖。此外，我们还介绍了 AfriE5，通过跨语言对比蒸馏对指令调优的 mE5 模型进行适应，以适应非洲语言。我们的评估显示，AfriE5 在性能上达到了最先进的水平，优于 Gemini-Embeddings 和 mE5 等强大基线。
英文摘要：
Text embeddings are an essential building component of several NLP tasks such as retrieval-augmented generation which is crucial for preventing hallucinations in LLMs. Despite the recent release of massively multilingual MTEB (MMTEB), African languages remain underrepresented, with existing tasks often repurposed from translation benchmarks such as FLORES clustering or SIB-200. In this paper, we introduce AfriMTEB -- a regional expansion of MMTEB covering 59 languages, 14 tasks, and 38 datasets, including six newly added datasets. Unlike many MMTEB datasets that include fewer than five languages, the new additions span 14 to 56 African languages and introduce entirely new tasks, such as hate speech detection, intent detection, and emotion classification, which were not previously covered. Complementing this, we present AfriE5, an adaptation of the instruction-tuned mE5 model to African languages through cross-lingual contrastive distillation. Our evaluation shows that AfriE5 achieves state-of-the-art performance, outperforming strong baselines such as Gemini-Embeddings and mE5.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
