同行评分中的偏差消除与精确度激励
Removing Bias and Incentivizing Precision in Peer-grading
摘要:
我们研究了具有竞争力的评分者的同伴评分,当他们的同伴得分较低时,他们享有较高的效用。我们提出了一个新的机制,PEQA,激励这样的评分员通过一个分数分配规则,聚合从多个同行评价的最终分数,和一个分级性能分数,奖励表现在同行分级练习。PEQA使年级偏见无关紧要。此外,在PEQA下,一个同龄人的效用随着她评分的可靠性而单调增加,而不管她的竞争力和她的同事的行为如何。在一个合理的一般类的分数分配规则,PEQA唯一满足这种效用-可靠性单调性。当评分是昂贵的,成本是私人信息,修改后的版本PEQA实现了社会最优的努力选择的同行评价游戏的均衡。从我们的课堂实验数据证实了我们的理论假设,并表明PEQA优于流行的中位数机制。
英文摘要:
We study peer-grading with competitive graders who enjoy a higher utility when their peers get lower scores. We propose a new mechanism, PEQA, that incentivizes such graders through a score-assignment rule which aggregates the final score from multiple peer-evaluations, and a grading performance score that rewards performance in the peer-grading exercise. PEQA makes grader-bias irrelevant. Additionally, under PEQA, a peer-grader's utility increases monotonically with the reliability of her grading, irrespective of her competitiveness and how her co-graders act. In a reasonably general class of score assignment rules, PEQA uniquely satisfies this utility- reliability monotonicity. When grading is costly and costs are private information, a modified version of PEQA implements the socially optimal effort choices in an equilibrium of the peer-evaluation game. Data from our classroom experiments confirm our theoretical assumptions and show that PEQA outperforms the popular median mechanism.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。