arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.00977cs.IRcs.CL
收藏

面向多语言信息检索的蒸馏方法

Distillation for Multilingual Information Retrieval

Johns Hopkins University(约翰斯·霍普金斯大学)

Eugene Yang, Dawn Lawrie, James Mayfield

AI总结:

本文提出多语言Translate-Distill(MTD)方法,将Translate-Distill扩展至多语言信息检索,使ColBERT-X能对多语言文档分配可比相关性分数,并在nDCG@20和MAP上显著超越此前的Multilingual Translate-Train方法。

摘要:

近期在跨语言信息检索(CLIR)领域的研究中,查询和文档使用不同语言,已显示出 Translate-Distill 框架的优势,该框架利用翻译和蒸馏训练跨语言神经双编码器模型。然而,Translate-Distill 仅支持单一文档语言。多语言信息检索(MLIR)需要对多语言文档集合进行排序,其训练难度高于 CLIR,因为模型必须为不同语言的文档分配可比较的相关性分数。本工作扩展了 Translate-Distill,并提出了用于 MLIR 的多语言 Translate-Distill(MTD)。我们证明,使用 MTD 训练的 ColBERT-X 模型在 nDCG@20 上比使用 Multilingual Translate-Train(此前的先进训练方法)训练的对应模型高出 5% 至 25%,在 MAP 上高出 15% 至 45%。我们还表明该模型对训练批次中语言混合方式具有鲁棒性。我们的实现已在 GitHub 上公开。

英文摘要:

Recent work in cross-language information retrieval (CLIR), where queries and documents are in different languages, has shown the benefit of the Translate-Distill framework that trains a cross-language neural dual-encoder model using translation and distillation. However, Translate-Distill only supports a single document language. Multilingual information retrieval (MLIR), which ranks a multilingual document collection, is harder to train than CLIR because the model must assign comparable relevance scores to documents in different languages. This work extends Translate-Distill and propose Multilingual Translate-Distill (MTD) for MLIR. We show that ColBERT-X models trained with MTD outperform their counterparts trained ith Multilingual Translate-Train, which is the previous state-of-the-art training approach, by 5% to 25% in nDCG@20 and 15% to 45% in MAP. We also show that the model is robust to the way languages are mixed in training batches. Our implementation is available on GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑