使用单语知识库的多语言信息检索
Multilingual Information Retrieval with a Monolingual Knowledge Base
AI总结：
本文提出通过加权采样对比学习微调多语言嵌入模型，实现基于单语知识库的跨语言检索，并在MRR和Recall@3上取得显著提升，且方法语言无关。
摘要：
多语言信息检索已成为扩展跨语言知识共享的强大工具。另一方面，高质量知识库的资源往往稀缺且仅覆盖有限的语言，因此，一个有效的嵌入模型，能够将不同语言的句子转换到与知识库语言相同的特征向量空间，成为跨语言知识共享的关键要素，尤其是将高资源语言中可用的知识迁移到低资源语言时。在本文中，我们提出了一种新策略，通过加权采样对多语言嵌入模型进行对比学习微调，从而实现使用单语知识库的多语言信息检索。我们证明，与标准采样策略相比，加权采样策略在MRR上带来了最高31.03%的性能提升，在Recall@3上带来了最高33.98%的性能提升。此外，我们提出的方法具有语言无关性，适用于多语言和语码转换两种使用场景。
英文摘要：
Multilingual information retrieval has emerged as powerful tools for expanding knowledge sharing across languages. On the other hand, resources on high quality knowledge base are often scarce and in limited languages, therefore an effective embedding model to transform sentences from different languages into a feature vector space same as the knowledge base language becomes the key ingredient for cross language knowledge sharing, especially to transfer knowledge available in high-resource languages to low-resource ones. In this paper we propose a novel strategy to fine-tune multilingual embedding models with weighted sampling for contrastive learning, enabling multilingual information retrieval with a monolingual knowledge base. We demonstrate that the weighted sampling strategy produces performance gains compared to standard ones by up to 31.03\% in MRR and up to 33.98\% in Recall@3. Additionally, our proposed methodology is language agnostic and applicable for both multilingual and code switching use cases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
