Translate-Distill：通过翻译与蒸馏学习跨语言密集检索
Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillation
AI总结：
本文提出Translate-Distill方法，利用单语或CLIR cross-encoder作为教师模型，通过知识蒸馏训练跨语言信息检索的双编码器学生模型，解决了CLIR缺乏大规模跨语言训练数据的问题。
摘要：
先前关于英语单语检索的工作表明，使用大量查询-文档对相关性判断训练的cross-encoder可以作为教师模型，训练出更高效但效果相似的双编码器（dual-encoder）学生模型。由于查询和文档语言不同时缺乏足够大的训练集，将类似知识蒸馏方法应用于训练跨语言信息检索（CLIR）的高效双编码器模型具有挑战性。因此，CLIR的最新技术依赖于从大型英语MS MARCO训练集翻译查询、文档或两者，这种方法称为Translate-Train。本文提出了一种替代方案Translate-Distill，其中使用来自单语cross-encoder或CLIR cross-encoder的知识蒸馏来训练双编码器CLIR学生模型。这种更丰富的设计空间使教师模型能在优化设置下执行推理，同时直接为学生模型训练CLIR。训练好的模型和工件已在Huggingface上公开提供。
英文摘要：
Prior work on English monolingual retrieval has shown that a cross-encoder trained using a large number of relevance judgments for query-document pairs can be used as a teacher to train more efficient, but similarly effective, dual-encoder student models. Applying a similar knowledge distillation approach to training an efficient dual-encoder model for Cross-Language Information Retrieval (CLIR), where queries and documents are in different languages, is challenging due to the lack of a sufficiently large training collection when the query and document languages differ. The state of the art for CLIR thus relies on translating queries, documents, or both from the large English MS MARCO training set, an approach called Translate-Train. This paper proposes an alternative, Translate-Distill, in which knowledge distillation from either a monolingual cross-encoder or a CLIR cross-encoder is used to train a dual-encoder CLIR student model. This richer design space enables the teacher model to perform inference in an optimized setting, while training the student model directly for CLIR. Trained models and artifacts are publicly available on Huggingface.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
