arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1603.04953cs.IR
收藏

通过信息检索措施对专家进行排名的缓解程序

Mitigation Procedures to Rank Experts through Information Retrieval Measures

Matthieu Vergne

摘要:

为了找到专家,不同的方法建立排名的人,假设他们是排名的专业知识水平,并使用典型的信息检索(IR)的措施,以评估其有效性。然而,我们发现专家排名(i)往往是部分有序的,(ii)不完整的,(iii)因此提供更多的顺序,而不是绝对的排名,这不是通常的IR措施所利用的。为了改善这种状态的艺术,我们建议修改IR中使用的形式主义,设计适当的措施比较专家的排名。在这份报告中,我们调查的第一步,提供缓解程序的三个问题,我们分析IR措施的帮助下,这些程序,以确定有趣的修订和剩余的限制。从这个分析中,我们看到,由于我们的缓解程序,大多数措施可以用于这个更通用的上下文。此外,基于精确度和召回率的措施,通常无法考虑排名项目的顺序,如果我们将排名表示为一组有序对,则首先感兴趣。另一方面,累积测量是专门为考虑顺序而设计的,但具有更高的复杂性,从而激励使用具有正确表示的精确/召回测量。

英文摘要:

In order to find experts, different approaches build rankings of people, assuming that they are ranked by level of expertise, and use typical Information Retrieval (IR) measures to evaluate their effectiveness. However, we figured out that expert rankings (i) tend to be partially ordered, (ii) incomplete, and (iii) consequently provide more an order rather than absolute ranks, which is not what usual IR measures exploit. To improve this state of the art, we propose to revise the formalism used in IR to design proper measures for comparing expert rankings. In this report, we investigate a first step by providing mitigation procedures for the three issues, and we analyse IR measures with the help of these procedures to identify interesting revisions and remaining limitations. From this analysis, we see that most of the measures can be exploited for this more generic context because of our mitigation procedures. Moreover, measures based on precision and recall, usually unable to consider the order of the ranked items, are of first interest if we represent a ranking as a set of ordered pairs. Cumulative measures, on the other hand, are specifically designed for considering the order but suffer from a higher complexity, motivating the use of precision/recall measures with the right representation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑