面向俄语的嵌入模型探索:ruMTEB基准与俄语嵌入模型设计
The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design
AI总结:
本文提出俄语嵌入模型ru-en-RoSBERTa及俄语版MTEB基准ruMTEB,涵盖七类任务,实验表明该模型在俄语任务上达到最先进水平,并开源模型、代码与排行榜。
摘要:
嵌入模型在自然语言处理(NLP)中发挥着关键作用,它们生成的文本嵌入被用于信息检索和语义文本相似度评估等多种任务。本文聚焦于俄语嵌入模型的相关研究,提出了一个面向俄语的新型嵌入模型ru-en-RoSBERTa,以及ruMTEB基准——这是大规模文本嵌入基准(Massive Text Embedding Benchmark,MTEB)的俄语扩展版本。该基准包含七类任务,如语义文本相似度、文本分类、重排序和检索。研究还在所提出的基准上评估了一组具有代表性的俄语模型和多语言模型。结果表明,新模型在俄语任务上取得了与最先进模型相当的效果。我们发布了ru-en-RoSBERTa模型,ruMTEB框架随附开源代码,并已集成到原始框架中,同时提供了公开排行榜。
英文摘要:
Embedding models play a crucial role in Natural Language Processing (NLP) by creating text embeddings used in various tasks such as information retrieval and assessing semantic text similarity. This paper focuses on research related to embedding models in the Russian language. It introduces a new Russian-focused embedding model called ru-en-RoSBERTa and the ruMTEB benchmark, the Russian version extending the Massive Text Embedding Benchmark (MTEB). Our benchmark includes seven categories of tasks, such as semantic textual similarity, text classification, reranking, and retrieval.The research also assesses a representative set of Russian and multilingual models on the proposed benchmark. The findings indicate that the new model achieves results that are on par with state-of-the-art models in Russian. We release the model ru-en-RoSBERTa, and the ruMTEB framework comes with open-source code, integration into the original framework and a public leaderboard.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。