arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.10173cs.CLcs.AIcs.IR
收藏

jina-embeddings-v3:具有任务LoRA的多语言嵌入模型

jina-embeddings-v3: Multilingual Embeddings With Task LoRA

Jina AI GmbH(吉纳人工智能有限公司)

Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, Han Xiao

AI总结:

jina-embeddings-v3是一款5.7亿参数的多语言嵌入模型,通过任务LoRA适配器在MTEB基准上超越OpenAI和Cohere,支持8192令牌上下文和可降至32维的嵌入。

摘要:

我们推出了jina-embeddings-v3,这是一种新型文本嵌入模型,拥有5.7亿参数,在多语言数据和长上下文检索任务上达到了最先进的性能,支持长达8192个令牌的上下文长度。该模型包含一组任务特定的低秩自适应(LoRA)适配器,用于生成查询-文档检索、聚类、分类和文本匹配的高质量嵌入。在MTEB基准上的评估显示,jina-embeddings-v3在英语任务上优于OpenAI和Cohere最新的专有嵌入模型,同时在所有多语言任务上均优于multilingual-e5-large-instruct。默认输出维度为1024,用户可以通过Matryoshka表示学习灵活地将嵌入维度降至最低32,而不影响性能。

英文摘要:

We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks. With a default output dimension of 1024, users can flexibly reduce the embedding dimensions to as low as 32 without compromising performance, enabled by Matryoshka Representation Learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑