arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14318cs.CL
收藏

评估波兰语文本排序模型的泛化能力

Assessing generalization capability of text ranking models in Polish

National Information Processing Institute(国家信息处理研究所)

Sławomir Dadas, Małgorzata Grębowiec

AI总结:

本文针对波兰语重排序任务,基于41个信息检索基准全面评估现有及自训模型,发现多数模型域外泛化困难,并通过有效优化与大规模数据训练出紧凑且泛化能力强的新最优模型。

摘要:

检索增强生成(RAG)正成为一种日益流行的技术,用于将内部知识库与大型语言模型集成。在典型的RAG流程中,使用三个模型,分别负责检索、重排序和生成阶段。本文聚焦于波兰语的重排序问题,考察重排序器的性能,并将其结果与可用的检索模型进行比较。我们利用一个包含41个多样化波兰语信息检索任务的基准,对现有模型以及我们训练的模型进行了全面评估。实验结果表明,大多数模型在域外泛化方面存在困难。然而,有效的优化方法与大规模训练数据集的结合,使得构建既紧凑又具备泛化能力的重排序器成为可能。我们最好的模型为波兰语重排序建立了新的最优水平(state-of-the-art),超越了参数规模多达30倍的现有模型。

英文摘要:

Retrieval-augmented generation (RAG) is becoming an increasingly popular technique for integrating internal knowledge bases with large language models. In a typical RAG pipeline, three models are used, responsible for the retrieval, reranking, and generation stages. In this article, we focus on the reranking problem for the Polish language, examining the performance of rerankers and comparing their results with available retrieval models. We conduct a comprehensive evaluation of existing models and those trained by us, utilizing a benchmark of 41 diverse information retrieval tasks for the Polish language. The results of our experiments show that most models struggle with out-of-domain generalization. However, a combination of effective optimization method and a large training dataset allows for building rerankers that are both compact in size and capable of generalization. The best of our models establishes a new state-of-the-art for reranking in the Polish language, outperforming existing models with up to 30 times more parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑