基于维基百科的俄语信息检索基准 RusBEIR 数据集
Wikipedia-based Datasets in Russian Information Retrieval Benchmark RusBEIR
AI总结：
本文基于俄语维基百科“你知道吗”栏目构建新数据集，扩展 RusBEIR 基准，通过对比词汇与神经检索模型并引入重排序，提升俄语信息检索性能。
摘要：
在本文中，我们提出了一系列新颖的俄语信息检索数据集，这些数据集构建自俄语维基百科的“你知道吗……”栏目。我们的数据集通过利用有趣的事实及其在句子级别标注分级相关性的参考维基百科文章，支持一系列检索任务，包括事实核查、检索增强生成和全文检索。我们描述了数据集创建的方法，该方法能够扩展现有的俄语信息检索（IR）资源。通过大量实验，我们扩展了 RusBEIR 研究，将 BM25 等词汇检索模型与针对俄语微调的最先进神经架构以及多语言模型进行了比较。实验结果表明，词汇方法在全文检索上往往优于神经模型，而神经方法在较短文本中更能捕捉词汇语义，例如在事实核查或细粒度检索中。利用我们新创建的数据集，我们还分析了文档长度对检索性能的影响，并证明将检索与神经重排序相结合能够持续改善结果。我们的贡献扩展了可用于俄语信息检索研究的资源，并强调了准确评估检索模型以实现最佳性能的重要性。所有数据集均在 HuggingFace 上公开提供。为便于复现和未来研究，我们还在 GitHub 上发布了完整实现。
英文摘要：
In this paper, we present a novel series of Russian information retrieval datasets constructed from the "Did you know..." section of Russian Wikipedia. Our datasets support a range of retrieval tasks, including fact-checking, retrieval-augmented generation, and full-document retrieval, by leveraging interesting facts and their referenced Wikipedia articles annotated at the sentence level with graded relevance. We describe the methodology for dataset creation that enables the expansion of existing Russian Information Retrieval (IR) resources. Through extensive experiments, we extend the RusBEIR research by comparing lexical retrieval models, such as BM25, with state-of-the-art neural architectures fine-tuned for Russian, as well as multilingual models. Results of our experiments show that lexical methods tend to outperform neural models on full-document retrieval, while neural approaches better capture lexical semantics in shorter texts, such as in fact-checking or fine-grained retrieval. Using our newly created datasets, we also analyze the impact of document length on retrieval performance and demonstrate that combining retrieval with neural reranking consistently improves results. Our contribution expands the resources available for Russian information retrieval research and highlights the importance of accurate evaluation of retrieval models to achieve optimal performance. All datasets are publicly available at HuggingFace. To facilitate reproducibility and future research, we also release the full implementation on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
