jina-embeddings-v3: Multilingual Embeddings With Task LoRA
Jina AI GmbH(吉纳人工智能有限公司)
Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, Han Xiao
备注:
20 pages, pp11-13 references, pp14-20 appendix and experiment tables
We introduce jina-embeddings-v3, a novel text embedding model with 570 million parameters, achieves state-of-the-art performance on multilingual data and long-context retrieval tasks, supporting context lengths of up to 8192 tokens. The model includes a set of task-specific Low-Rank Adaptation (LoRA) adapters to generate high-quality embeddings for query-document retrieval, clustering, classification, and text matching. Evaluation on the MTEB benchmark shows that jina-embeddings-v3 outperforms the latest proprietary embeddings from OpenAI and Cohere on English tasks, while achieving superior performance compared to multilingual-e5-large-instruct across all multilingual tasks. With a default output dimension of 1024, users can flexibly reduce the embedding dimensions to as low as 32 without compromising performance, enabled by Matryoshka Representation Learning.
Efficient Multilingual ASR Finetuning via LoRA Language Experts
Auditory Cognition and Computational Acoustics Lab(听觉认知与计算声学实验室);MoE Key Lab of Artificial Intelligence(人工智能MoE重点实验室);AI Institute School of Computer Science, Shanghai Jiao Tong University(计算机学院人工智能研究所,上海交通大学);Tencent AI Lab(腾讯AI实验室)
Seamless Language Expansion: Enhancing Multilingual Mastery in Self-Supervised Models
Department of Systems Engineering and Engineering Management(系统工程与工程管理系);The Chinese University of Hong Kong(香港中文大学);Centre for Perceptual and Interactive Intelligence (CPII) Limited(感知与交互智能中心(CPII)有限公司)
本文提出将 LoRA 集成到已有自监督模型并结合数据组合与重新聚类保留策略,使 mHuBERT 高效扩展至普通话,在语音重合成任务中显著提升 MOS 并降低 WER,同时保持原有语言能力不退化。
OrthoGeoLoRA: Geometric Parameter-Efficient Fine-Tuning for Structured Social Science Concept Retrieval on theWeb
University of Surrey(萨里大学);Washington State University(华盛顿州立大学);University of Oxford(牛津大学);New York University Shanghai(纽约大学上海分校);University College London(伦敦大学学院)
Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task
Department of Information Systems, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania(信息系统系,信息学院,凯纳斯技术大学,凯纳斯,立陶宛);Centre of Information Systems Design Technologies, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania(信息系统设计技术中心,信息学院,凯纳斯技术大学,凯纳斯,立陶宛)
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
Tsinghua University(清华大学);Beijing Information Science and Technology University(北京信息科技大学);National University of Singapore(新加坡国立大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Beijing University of Posts and Telecommunications(北京邮电大学)
Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition
Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University(智能教育与大数据上海工程研究中心,上海师范大学);SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University(上海师范大学Unisound自然人机交互实验室);Unisound AI Technology Co., Ltd.(Unisound人工智能技术有限公司)
Evaluating Multilingual Sentence Embeddings for Translation Error Detection:An English--Greek Contrastive Study
Ionian University(爱奥尼亚大学);Laboratory on Digital Libraries and Electronic Publishing(数字图书馆与电子出版实验室);Centre for Literary Translation and Book Policy(文学翻译与图书政策中心)
Benchmarking Parameter-Efficient Fine-Tuning of Large Language Models for Low-Resource Tajik Text Generation with the Tajik Web Corpus
Institute of Computational Mathematics and Information Technologies, Kazan Federal University(喀山联邦大学计算数学与信息技术研究所);Bokhtar State University named after Nosiri Khusrav(努西里·胡斯劳夫命名的博赫塔国立大学)