arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15059cs.CLcs.IR
收藏

ColBERT-XM:一种用于零样本多语言信息检索的模块化多向量表示模型

ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval

Maastricht University(马斯特里赫特大学)

Antoine Louis, Vageesh Saxena, Gijs van Dijck, Gerasimos Spanakis

AI总结:

ColBERT-XM 提出一种模块化多向量稠密检索模型,仅利用单一高资源语言数据实现零样本多语言迁移,在保持竞争力的同时提升数据效率并降低能耗。

摘要:

最先进的神经检索器主要关注英语等高资源语言,这阻碍了它们在涉及其他语言的检索场景中的应用。当前方法通过利用能够进行跨语言迁移的多语言预训练语言模型来规避非英语语言中缺乏高质量标注数据的问题。然而,这些模型需要在多种语言上进行大量的任务特定微调,在预训练语料库中表示极少的语言上通常表现不佳,并且难以在预训练阶段之后纳入新语言。在这项工作中,我们提出了一种新颖的模块化稠密检索模型,该模型从单一高资源语言的丰富数据中学习,并有效地零样本迁移到广泛的语言,从而消除了对语言特定标注数据的需求。我们的模型 ColBERT-XM 在与在多种语言上使用更广泛数据集训练的现有最先进多语言检索器的比较中表现出具有竞争力的性能。进一步分析表明,我们的模块化方法具有很高的数据效率,能有效适应分布外数据,并显著降低能耗和碳排放。通过展示其在零样本场景中的熟练能力,ColBERT-XM 标志着向更可持续和更具包容性的检索系统的转变,使众多语言中的有效信息获取成为可能。我们公开发布代码和模型供社区使用。

英文摘要:

State-of-the-art neural retrievers predominantly focus on high-resource languages like English, which impedes their adoption in retrieval scenarios involving other languages. Current approaches circumvent the lack of high-quality labeled data in non-English languages by leveraging multilingual pretrained language models capable of cross-lingual transfer. However, these models require substantial task-specific fine-tuning across multiple languages, often perform poorly in languages with minimal representation in the pretraining corpus, and struggle to incorporate new languages after the pretraining phase. In this work, we present a novel modular dense retrieval model that learns from the rich data of a single high-resource language and effectively zero-shot transfers to a wide array of languages, thereby eliminating the need for language-specific labeled data. Our model, ColBERT-XM, demonstrates competitive performance against existing state-of-the-art multilingual retrievers trained on more extensive datasets in various languages. Further analysis reveals that our modular approach is highly data-efficient, effectively adapts to out-of-distribution data, and significantly reduces energy consumption and carbon emissions. By demonstrating its proficiency in zero-shot scenarios, ColBERT-XM marks a shift towards more sustainable and inclusive retrieval systems, enabling effective information accessibility in numerous languages. We publicly release our code and models for the community.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑