jina-embeddings-v5-text：任务导向的嵌入蒸馏
jina-embeddings-v5-text: Task-Targeted Embedding Distillation
AI总结：
本文提出结合模型蒸馏与任务特定对比损失的训练方法，生成紧凑高效嵌入模型，实验表明其在小型模型训练中效果优于纯对比或蒸馏方法，模型在多语言长文本处理中表现稳健。
摘要：
文本嵌入模型广泛用于语义相似性任务，包括信息检索、聚类和分类。通用模型通常通过单阶段或多阶段过程使用对比损失函数进行训练。我们引入了一种结合模型蒸馏技术与任务特定对比损失的新训练方法，生成紧凑且高性能的嵌入模型。我们的发现表明，这种方法在训练小型模型时比纯对比或蒸馏训练方法更有效。所生成的模型jina-embeddings-v5-text-small和jina-embeddings-v5-text-nano的基准得分超过或达到类似规模模型的最先进水平。jina-embeddings-v5-text模型还支持多种语言的长文本（最多32k个标记），并在截断和二进制量化下生成的嵌入保持稳健。模型权重已公开，希望进一步推动嵌入模型的发展。
英文摘要：
Text embedding models are widely used for semantic similarity tasks, including information retrieval, clustering, and classification. General-purpose models are typically trained with single- or multi-stage processes using contrastive loss functions. We introduce a novel training regimen that combines model distillation techniques with task-specific contrastive loss to produce compact, high-performance embedding models. Our findings suggest that this approach is more effective for training small models than purely contrastive or distillation-based training paradigms alone. Benchmark scores for the resulting models, jina-embeddings-v5-text-small and jina-embeddings-v5-text-nano, exceed or match the state-of-the-art for models of similar size. jina-embeddings-v5-text models additionally support long texts (up to 32k tokens) in many languages, and generate embeddings that remain robust under truncation and binary quantization. Model weights are publicly available, hopefully inspiring further advances in embedding model development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
