arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.14837cs.IRcs.LG
收藏

NeuralSearchX:以低成本为多语言元数据提供数十亿参数的重排序器

NeuralSearchX: Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost

NeuralMind

Thales Sales Almeida, Thiago Laitz, João Seródio, Luiz Henrique Bonifacio, Roberto Lotufo, Rodrigo Nogueira

摘要:

搜索API(免费和商业)的广泛可用性为元搜索引擎带来了增加覆盖范围和搜索结果质量的希望,同时降低了爬行和索引基础设施的维护成本。然而,合并策略通常包括需要仔细调整的复杂管道,这在文献中经常被忽视。在这项工作中,我们描述了NeuralSearchX,一个元搜索引擎的基础上,一个多用途的大型重排序模型,合并结果和突出的句子。由于我们架构的同质性,我们可以将优化工作集中在单个组件上。我们将我们的系统与微软的生物医学搜索进行比较,结果表明,我们的设计选择导致了一个具有竞争力的QPS的更具成本效益的系统,同时在广泛的公共基准测试中获得了接近最先进的结果。对两个特定领域任务的人工评估表明,我们的检索系统在nDCG@10分数方面大大优于Google API。通过详细描述我们的体系结构和实现,我们希望社区将建立在我们的设计选择之上。该系统可在https://neuralsearchx.nsx.ai上获得。

英文摘要:

The widespread availability of search API's (both free and commercial) brings the promise of increased coverage and quality of search results for metasearch engines, while decreasing the maintenance costs of the crawling and indexing infrastructures. However, merging strategies frequently comprise complex pipelines that require careful tuning, which is often overlooked in the literature. In this work, we describe NeuralSearchX, a metasearch engine based on a multi-purpose large reranking model to merge results and highlight sentences. Due to the homogeneity of our architecture, we could focus our optimization efforts on a single component. We compare our system with Microsoft's Biomedical Search and show that our design choices led to a much cost-effective system with competitive QPS while having close to state-of-the-art results on a wide range of public benchmarks. Human evaluation on two domain-specific tasks shows that our retrieval system outperformed Google API by a large margin in terms of nDCG@10 scores. By describing our architecture and implementation in detail, we hope that the community will build on our design choices. The system is available at https://neuralsearchx.nsx.ai.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑