arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.05917cs.IR
收藏

SpaDE:利用双文档编码器改进稀疏表示以用于第一阶段检索

SpaDE: Improving Sparse Representations using a Dual Document Encoder for First-stage Retrieval

Eunseong Choi, Sunkyung Lee, Minjin Choi, Hyeseon Ko, Young-In Song, Jongwuk Lee

AI总结:

针对稀疏检索中词汇不匹配与查询推理成本的权衡,提出双文档编码器模型 SpaDE 及协同训练策略,在多个基准上优于现有单编码器排序模型。

摘要:

稀疏文档表示已被广泛用于通过精确词汇匹配来检索相关文档。得益于预先计算的倒排索引,它支持快速的即席搜索,但会引发词汇不匹配问题。尽管近期使用预训练语言模型的神经排序模型能够解决这一问题,但它们通常需要高昂的查询推理成本,这意味着有效性与效率之间存在权衡。为应对这一权衡,我们提出了一种新颖的单编码器排序模型 Sparse retriever using a Dual document Encoder(SpaDE),通过双编码器学习文档表示。每个编码器都在以下两方面发挥核心作用:(i)调整词项的重要性以改进词汇匹配;(ii)扩展额外词项以支持语义匹配。此外,我们的协同训练策略能够有效训练双编码器,并避免在彼此训练过程中产生不必要的干预。在多个基准上的实验结果表明,SpaDE 优于现有的单编码器排序模型。

英文摘要:

Sparse document representations have been widely used to retrieve relevant documents via exact lexical matching. Owing to the pre-computed inverted index, it supports fast ad-hoc search but incurs the vocabulary mismatch problem. Although recent neural ranking models using pre-trained language models can address this problem, they usually require expensive query inference costs, implying the trade-off between effectiveness and efficiency. Tackling the trade-off, we propose a novel uni-encoder ranking model, Sparse retriever using a Dual document Encoder (SpaDE), learning document representation via the dual encoder. Each encoder plays a central role in (i) adjusting the importance of terms to improve lexical matching and (ii) expanding additional terms to support semantic matching. Furthermore, our co-training strategy trains the dual encoder effectively and avoids unnecessary intervention in training each other. Experimental results on several benchmarks show that SpaDE outperforms existing uni-encoder ranking models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑