arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1704.08464cs.AI
收藏

排名的一致性衡量

Consensus measure of rankings

School of Computing and Mathematics, Ulster University(阿尔斯特大学计算与数学学院)

Zhiwei Lin, Yi Li, Xiaolian Guo

摘要:

排名是项目的有序序列,其中具有较高排名分数的项目比具有较低排名分数的项目更优选。在许多信息系统中,排名被广泛用于表示对一组项目或候选项的偏好。排名的一致性度量是如何评估排名一致程度的问题。在许多信息系统中,一致性度量可用于评估排名,因为通常没有可用于评估的基本事实。文章提出了一种基于图表示的排序一致性度量方法,其中顶点或节点表示排序中的项目,边表示排序中项目之间的关系。这样的表示导致各种算法的一致性措施在不同方面的排名,包括共同模式的数量,固定长度的共同模式的数量和最长的共同模式的长度。建议的方法可用于各种排名,例如全面排名、部分排名和并列排名。本文演示了如何使用所提出的方法来评估排名聚合的质量和质量的顶级$k$排名从谷歌和必应搜索引擎。

英文摘要:

A ranking is an ordered sequence of items, in which an item with higher ranking score is more preferred than the items with lower ranking scores. In many information systems, rankings are widely used to represent the preferences over a set of items or candidates. The consensus measure of rankings is the problem of how to evaluate the degree to which the rankings agree. The consensus measure can be used to evaluate rankings in many information systems, as quite often there is not ground truth available for evaluation. This paper introduces a novel approach for consensus measure of rankings by using graph representation, in which the vertices or nodes are the items and the edges are the relationship of items in the rankings. Such representation leads to various algorithms for consensus measure in terms of different aspects of rankings, including the number of common patterns, the number of common patterns with fixed length and the length of the longest common patterns. The proposed measure can be adopted for various types of rankings, such as full rankings, partial rankings and rankings with ties. This paper demonstrates how the proposed approaches can be used to evaluate the quality of rank aggregation and the quality of top-$k$ rankings from Google and Bing search engines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑