多样性的好处:结合比较与评分以实现高效的评分
The Benefits of Diversity: Combining Comparisons and Ratings for Efficient Scoring
AI总结:
SCoRa通过结合比较和评分提升评分效率,实验证明其在模型不匹配情况下仍能准确恢复评分。
摘要:
人类应被要求单独评估实体还是进行比较评估?这个问题长期以来一直存在争议。在本工作中,我们展示了,有趣的是,结合这两种偏好获取方式可以优于只关注单一类型。更具体地说,我们介绍了SCoRa(通过比较和评分进行评分),一种统一的概率模型,允许从两种信号中学习。我们证明了SCoRa的MAP估计器具有良好的性质。它验证了单调性和鲁棒性保证。然后我们实证地表明,即使在模型不匹配的情况下,SCoRa也能恢复准确的评分。最有趣的是,我们识别了一个现实场景,其中结合比较和评分优于单独使用其中一种,尤其是在准确排序顶部实体至关重要的情况下。鉴于多种形式的信号实际上可用,SCoRa还为偏好学习提供了灵活的基础。
英文摘要:
Should humans be asked to evaluate entities individually or comparatively? This question has been the subject of long debates. In this work, we show that, interestingly, combining both forms of preference elicitation can outperform the focus on a single kind. More specifically, we introduce SCoRa (Scoring from Comparisons and Ratings), a unified probabilistic model that allows to learn from both signals. We prove that the MAP estimator of SCoRa is well-behaved. It verifies monotonicity and robustness guarantees. We then empirically show that SCoRa recovers accurate scores, even under model mismatch. Most interestingly, we identify a realistic setting where combining comparisons and ratings outperforms using either one alone, and when the accurate ordering of top entities is critical. Given the de facto availability of signals of multiple forms, SCoRa additionally offers a versatile foundation for preference learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。