arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.14534cs.IRcs.CL
收藏

Spacerini:基于Pyserini和Hugging Face的即插即用搜索引擎

Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face

Leipzig University(莱比锡大学);University of Waterloo(滑铁卢大学);Hugging Face;Sapienza University(萨皮恩扎大学)

Christopher Akiki, Odunayo Ogundepo, Aleksandra Piktus, Xinyu Zhang, Akintunde Oladipo, Jimmy Lin, Martin Potthast

AI总结:

本文提出Spacerini工具,集成Pyserini与Hugging Face,实现交互式搜索引擎的无缝构建与部署,降低非IR从业者使用稀疏和稠密检索模型的门槛,并展示了13个不同用例的搜索引擎。

摘要:

我们提出了Spacerini,这是一个将用于可复现信息检索研究的Pyserini工具包与Hugging Face相集成的工具,旨在实现交互式搜索引擎的无缝构建与部署。Spacerini让非IR(信息检索)从业者也能更轻松地使用最先进的稀疏和稠密检索模型,同时将部署工作量降至最低。这对于希望通过分析训练语料库来进行定性分析,从而更好地理解和验证其研究的NLP(自然语言处理)研究人员、希望展示集成到不断发展的Pyserini生态系统中的新检索模型的IR研究人员,以及复现其他研究人员工作的第三方都非常有用。Spacerini是开源的,包含用于在本地和远程加载、预处理、索引和部署搜索引擎的实用工具。我们展示了使用Spacerini为不同用例创建的13个搜索引擎的组合。

英文摘要:

We present Spacerini, a tool that integrates the Pyserini toolkit for reproducible information retrieval research with Hugging Face to enable the seamless construction and deployment of interactive search engines. Spacerini makes state-of-the-art sparse and dense retrieval models more accessible to non-IR practitioners while minimizing deployment effort. This is useful for NLP researchers who want to better understand and validate their research by performing qualitative analyses of training corpora, for IR researchers who want to demonstrate new retrieval models integrated into the growing Pyserini ecosystem, and for third parties reproducing the work of other researchers. Spacerini is open source and includes utilities for loading, preprocessing, indexing, and deploying search engines locally and remotely. We demonstrate a portfolio of 13 search engines created with Spacerini for different use cases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑