arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1607.03401cs.HCcs.LGcs.MM
收藏

用于众包偏好聚集的简约混合效应HodgeRank

Parsimonious Mixed-Effects HodgeRank for Crowdsourced Preference Aggregation

State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences(信息安全国家重点实验室,中国科学院信息工程研究所);Peking University(北京大学)

Qianqian Xu, Jiechao Xiong, Xiaochun Cao, Yuan Yao

摘要:

在众包偏好聚合中,通常假设所有注释者都服从于一个共同的偏好或效用函数,该函数在实验中产生他们的比较行为。然而,在现实中,注释器受到由于多标准,异常,或这些行为的混合的变化。在本文中,我们提出了一个基于HodgeRank的简约混合效应模型,它既考虑了大多数注释者遵循共同的线性效用模型的固定效应,也考虑了一小部分注释者可能会显著偏离共同的线性效用模型并表现出强烈的个性化偏好的随机效应。HodgeRank已成功应用于多媒体的主观质量评价,并解决了成对的众包排名数据的全球共识排名和循环的利益冲突。作为一种扩展,我们提出的方法进一步探讨了利益冲突,通过随机效应在注释者的具体变化。本文的关键算法建立了一个动态的路径,从共同的效用到个人的变化,不同程度的简约或稀疏的个性化,基于新开发的线性化Bregman算法与逆尺度空间方法。最后通过仿真实例和三个真实的众包数据集的实验验证了该方法的有效性,实验结果表明,与HodgeRank相比,该方法具有更好的性能(即更小的测试误差).

英文摘要:

In crowdsourced preference aggregation, it is often assumed that all the annotators are subject to a common preference or utility function which generates their comparison behaviors in experiments. However, in reality annotators are subject to variations due to multi-criteria, abnormal, or a mixture of such behaviors. In this paper, we propose a parsimonious mixed-effects model based on HodgeRank, which takes into account both the fixed effect that the majority of annotators follows a common linear utility model, and the random effect that a small subset of annotators might deviate from the common significantly and exhibits strongly personalized preferences. HodgeRank has been successfully applied to subjective quality evaluation of multimedia and resolves pairwise crowdsourced ranking data into a global consensus ranking and cyclic conflicts of interests. As an extension, our proposed methodology further explores the conflicts of interests through the random effect in annotator specific variations. The key algorithm in this paper establishes a dynamic path from the common utility to individual variations, with different levels of parsimony or sparsity on personalization, based on newly developed Linearized Bregman Algorithms with Inverse Scale Space method. Finally the validity of the methodology are supported by experiments with both simulated examples and three real-world crowdsourcing datasets, which shows that our proposed method exhibits better performance (i.e. smaller test error) compared with HodgeRank due to its parsimonious property.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑