论查询性能预测逐点评估的可行性与鲁棒性
On the Feasibility and Robustness of Pointwise Evaluation of Query Performance Prediction
AI总结:
针对现有列表式查询性能预测评估不支持单个查询评估的缺陷,提出逐点QPP框架,通过测量预测值与真实值偏差并聚合来评估单查询质量,实验证明该方法能减小评估方差。
摘要:
尽管查询的检索有效性彼此相互独立,但查询性能预测(QPP)系统的评估一直是通过测量整个查询集合上的排名相关性来进行的。这种列表式方法存在诸多缺点,尤其是它不支持评估单个查询的QPP这一常见需求。在本文中,我们提出了一种逐点QPP框架,该框架通过测量每个预测值与对应真实值之间的偏差,然后将结果在查询集合上进行聚合,从而允许我们评估QPP系统对单个查询的质量。我们的实验表明,这种新方法在一系列不同的目标指标和检索模型上,使得QPP评估的方差更小。
英文摘要:
Despite the retrieval effectiveness of queries being mutually independent of one another, the evaluation of query performance prediction (QPP) systems has been carried out by measuring rank correlation over an entire set of queries. Such a listwise approach has a number of disadvantages, notably that it does not support the common requirement of assessing QPP for individual queries. In this paper, we propose a pointwise QPP framework that allows us to evaluate the quality of a QPP system for individual queries by measuring the deviations between each prediction versus the corresponding true value, and then aggregating the results over a set of queries. Our experiments demonstrate that this new approach leads to smaller variances in QPP evaluations across a range of different target metrics and retrieval models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。