Surprise：通过极值理论进行结果列表截断
Surprise: Result List Truncation via Extreme Value Theory
摘要：
信息检索的工作主要集中在排序和相关性上：给定一个查询，返回一些按相关性排序的结果给用户。然而，结果列表截断的问题，或者在哪里截断结果的排名列表，尽管在各种应用中是至关重要的，但却受到较少的关注。这种截断是在结果的总体相关性或有用性与处理更多结果的用户成本之间的平衡行为。结果列表截断可能具有挑战性，因为相关性分数通常没有很好地校准。这在大规模IR系统中尤其如此，其中文档和查询嵌入在相同的度量空间中，并且查询的最近文档邻居在推理期间返回。在这里，相关性与查询和候选文档之间的距离成反比，但是构成相关性的距离因查询而异，并且随着更多文档添加到索引而动态变化。在这项工作中，我们提出了惊喜评分，一种统计方法，利用广义帕累托分布，在极值理论产生可解释的和校准的相关性分数在查询时只使用排名分数。我们证明了它在图像，文本和IR数据集的结果列表截断任务上的有效性，并将其与经典和最近的基线进行比较。我们将假设检验和$p$值联系起来。
英文摘要：
Work in information retrieval has largely been centered around ranking and relevance: given a query, return some number of results ordered by relevance to the user. The problem of result list truncation, or where to truncate the ranked list of results, however, has received less attention despite being crucial in a variety of applications. Such truncation is a balancing act between the overall relevance, or usefulness of the results, with the user cost of processing more results. Result list truncation can be challenging because relevance scores are often not well-calibrated. This is particularly true in large-scale IR systems where documents and queries are embedded in the same metric space and a query's nearest document neighbors are returned during inference. Here, relevance is inversely proportional to the distance between the query and candidate document, but what distance constitutes relevance varies from query to query and changes dynamically as more documents are added to the index. In this work, we propose Surprise scoring, a statistical method that leverages the Generalized Pareto distribution that arises in extreme value theory to produce interpretable and calibrated relevance scores at query time using nothing more than the ranked scores. We demonstrate its effectiveness on the result list truncation task across image, text, and IR datasets and compare it to both classical and recent baselines. We draw connections to hypothesis testing and $p$-values.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
