arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.09175cs.IRcs.AI
收藏

查询扩展能否提升强交叉编码器排序器的泛化能力?

Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?

University of Waterloo(滑铁卢大学);Google(谷歌公司)

Minghan Li, Honglei Zhuang, Kai Hui, Zhen Qin, Jimmy Lin, Rolf Jagerman, Xuanhui Wang, Michael Bendersky

AI总结:

本文针对查询扩展会损害强交叉编码器排序器的问题,提出通过提示工程生成高质量关键词并进行最小干扰查询修改,再以自一致性和倒数排名加权融合各扩展查询排序结果,在BEIR和TREC DL上提升了MonoT5和RankT5的nDCG@10,证明了查询扩展可改善强排序器的泛化能力。

摘要:

查询扩展已被广泛用于改善第一阶段检索器的搜索结果,但其对第二阶段交叉编码器排序器的影响仍未得到充分探索。Weller等人[44]最近的一项工作表明,当前的扩展技术对DPR和BM25等较弱模型有益,但会损害MonoT5等较强排序器的性能。在本文中,我们重新审视这一结论并提出以下问题:查询扩展能否提升强交叉编码器排序器的泛化能力?为回答该问题,我们首先将流行的查询扩展方法应用于最先进的交叉编码器排序器,并验证了零样本性能的下降。我们在实验中确定了交叉编码器的两个关键步骤:高质量关键词生成和最小干扰的查询修改。我们表明,通过提示工程和通过融合聚合每个扩展查询的排序结果,有可能提升强神经排序器的泛化能力。具体而言,我们首先调用一个遵循指令的语言模型,通过推理链生成关键词。利用自一致性和倒数排名加权,我们进一步动态组合每个扩展查询的排序结果。在BEIR和TREC Deep Learning 2019/2020上的实验表明,遵循这些步骤后,MonoT5和RankT5的nDCG@10分数均得到提升,这为将查询扩展应用于强交叉编码器排序器指明了一个方向。

英文摘要:

Query expansion has been widely used to improve the search results of first-stage retrievers, yet its influence on second-stage, cross-encoder rankers remains under-explored. A recent work of Weller et al. [44] shows that current expansion techniques benefit weaker models such as DPR and BM25 but harm stronger rankers such as MonoT5. In this paper, we re-examine this conclusion and raise the following question: Can query expansion improve generalization of strong cross-encoder rankers? To answer this question, we first apply popular query expansion methods to state-of-the-art cross-encoder rankers and verify the deteriorated zero-shot performance. We identify two vital steps for cross-encoders in the experiment: high-quality keyword generation and minimal-disruptive query modification. We show that it is possible to improve the generalization of a strong neural ranker, by prompt engineering and aggregating the ranking results of each expanded query via fusion. Specifically, we first call an instruction-following language model to generate keywords through a reasoning chain. Leveraging self-consistency and reciprocal rank weighting, we further combine the ranking results of each expanded query dynamically. Experiments on BEIR and TREC Deep Learning 2019/2020 show that the nDCG@10 scores of both MonoT5 and RankT5 following these steps are improved, which points out a direction for applying query expansion to strong cross-encoder rankers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑