打破平庸:一种通用的查询性能预测评估框架
Breaking Flat: A Generalised Query Performance Prediction Evaluation Framework
AI总结:
本文提出了一种通用的查询性能预测评估框架,通过三种不同设置评估排序模型的有效性,发现预测最佳排序器比预测查询难度更具挑战性。
摘要:
传统的查询性能预测(QPP)用例是确定给定排序模型下哪些查询表现良好,哪些表现差。更细致且可能更具挑战性的扩展是确定哪些排序模型对给定查询最有效。在本文中,我们将QPP任务及其评估推广到三种设置:(i)单排序器多查询(SRMQ-PP),对应于标准用例;(ii)多排序器单查询(MRSQ-PP),评估QPP模型选择最有效排序器的能力;(iii)多排序器多查询(MRMQ-PP),考虑所有查询排序器对的联合预测。我们的结果表明:(a)QPP模型的相对有效性在不同任务(SRMQ-PP vs. MRSQ-PP)中差异显著;(b)预测查询的最佳排序器比预测给定排序器的查询难度要困难得多。
英文摘要:
The traditional use-case of query performance prediction (QPP) is to identify which queries perform well and which perform poorly for a given ranking model. A more fine-grained and arguably more challenging extension of this task is to determine which ranking models are most effective for a given query. In this work, we generalize the QPP task and its evaluation into three settings: (i) SingleRanker MultiQuery (SRMQ-PP), corresponding to the standard use case; (ii) MultiRanker SingleQuery (MRSQ-PP), which evaluates a QPP model's ability to select the most effective ranker for a query; and (iii) MultiRanker MultiQuery (MRMQ-PP), which considers predictions jointly across all query ranker pairs. Our results show that (a) the relative effectiveness of QPP models varies substantially across tasks (SRMQ-PP vs. MRSQ-PP), and (b) predicting the best ranker for a query is considerably more difficult than predicting the relative difficulty of queries for a given ranker.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。