基于多向量模型构建更好的单语日语检索器
Towards Better Monolingual Japanese Retrievers with Multi-Vector Models
AI总结：
本文提出多向量检索器系列 JaColBERT，仅用 1.1 亿参数和更少数据训练，在所有数据集上超越现有单语日语检索器及域外任务上的多语言模型，证明了专用模型处理语言特异性的优势。
摘要：
由于与英语相比，特定语言的训练数据通常较为稀缺，许多语言的文档检索在很大程度上一直依赖多语言模型。在日语中，基于深度学习的最佳检索方法依赖多语言密集嵌入模型，而纯日语模型则远远落后。然而，多语言模型在训练时需要多得多的计算资源和数据，并且具有更高的计算和内存需求，同时常常遗漏文化相关信息。在本文中，我们引入了 JaColBERT，这是一系列多向量检索器，其训练数据比多语言对应模型少两个数量级，同时达到了具有竞争力的性能。我们最强的模型在所有数据集上大幅超越了所有现有的单语日语检索器，并且在所有域外任务上超越了最强的现有的多语言模型，这凸显了对能够处理语言特异性的专用模型的需求。这些结果是在仅使用 1.1 亿参数的模型上取得的，该模型明显小于所有多语言模型，并且仅使用有限的日语语言数据。我们相信，我们的结果展示了在广泛领域中支持日语检索增强应用流水线的巨大前景。
英文摘要：
As language-specific training data tends to be sparsely available compared to English, document retrieval in many languages has been largely relying on multilingual models. In Japanese, the best performing deep-learning based retrieval approaches rely on multilingual dense embedders, with Japanese-only models lagging far behind. However, multilingual models require considerably more compute and data to train and have higher computational and memory requirements while often missing out on culturally-relevant information. In this paper, we introduce JaColBERT, a family of multi-vector retrievers trained on two magnitudes fewer data than their multilingual counterparts while reaching competitive performance. Our strongest model largely outperform all existing monolingual Japanese retrievers on all dataset, as well as the strongest existing multilingual models on all out-of-domain tasks, highlighting the need for specialised models able to handle linguistic specificities. These results are achieved using a model with only 110 million parameters, considerably smaller than all multilingual models, and using only a limited Japanese-language. We believe our results show great promise to support Japanese retrieval-enhanced application pipelines in a wide variety of domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
