M3-Embedding：通过自知识蒸馏实现多语言、多功能、多粒度的文本嵌入
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
AI总结：
M3-Embedding通过自知识蒸馏实现多语言、多功能和多粒度的文本嵌入，提升多语言检索性能。
摘要：
在本文中，我们介绍了一种名为M3-Embedding的新嵌入模型，其特点是具有多语言、多功能和多粒度的 versatility。它为超过100种工作语言的语义检索提供了统一支持。它可以同时完成三种常见的检索功能：密集检索、多向量检索和稀疏检索。此外，它还能处理不同粒度的输入，从短句到多达8192个标记的长文档。M3-Embedding的有效训练提出了一系列技术贡献。值得注意的是，我们提出了一种新颖的自知识蒸馏方法，其中来自不同检索功能的相关性分数可以作为教师信号来提高训练质量。我们还优化了批处理策略，这使大批次大小和高训练吞吐量得以实现，从而提高嵌入的辨别性。M3-Embedding在我们的实验中表现出优越的性能，从而在多语言、跨语言和长文档检索基准上取得了新的最先进结果。
英文摘要：
In this paper, we introduce a new embedding model called M3-Embedding, which is distinguished for its versatility in \textit{Multi-Linguality}, \textit{Multi-Functionality}, and \textit{Multi-Granularity}. It provides a uniform support for the semantic retrieval of more than 100 working languages. It can simultaneously accomplish the three common retrieval functionalities: dense retrieval, multi-vector retrieval, and sparse retrieval. Besides, it is also capable of processing inputs of different granularities, spanning from short sentences to long documents of up to 8,192 tokens. The effective training of M3-Embedding presents a series of technical contributions. Notably, we propose a novel self-knowledge distillation approach, where the relevance scores from different retrieval functionalities can be integrated as the teacher signal to enhance the training quality. We also optimize the batching strategy, which enables a large batch size and high training throughput to improve the discriminativeness of embeddings. M3-Embedding exhibits a superior performance in our experiment, leading to new state-of-the-art results on multilingual, cross-lingual, and long-document retrieval benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
