arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1604.03178cs.GT
收藏

真实同伴评分的激励措施

Incentives for Truthful Peer Grading

Luca de Alfaro, Michael Shavlovsky, Vassilis Polychronopoulos

摘要:

只有当用户有动机如实评分时,同行评分系统才能有效。我们在课堂上观察到的一个不真实评分的例子是,学生给所有提交的材料都打最高分。采用简单的评分方案,例如平均分配的分数,所有学生都将获得最高分。在本文中,我们开发了三个分级计划,提供奖励真实的同行评分。在第一种方案中,教师对提交的分数p进行评分,并惩罚评分偏离教师评分的学生。我们提供的p的下限,以确保真实性,并得出结论,这些计划只适用于中等班级规模,最多几百名学生。为了克服这一限制,我们提出了一个分层扩展的监督计划,我们表明,它可以处理任何规模的类与有限的(和很少)教师的工作,因此适用于大规模开放式在线课程(MOOC)。最后,我们提出了无监督的激励计划,其中学生的激励是基于统计性质的成绩分布,没有任何分级所需的教师。我们表明,拟议的无监督计划提供了激励真实的评分,在价格可能是不公平的个别学生。

英文摘要:

Peer grading systems work well only if users have incentives to grade truthfully. An example of non-truthful grading, that we observed in classrooms, consists in students assigning the maximum grade to all submissions. With a naive grading scheme, such as averaging the assigned grades, all students would receive the maximum grade. In this paper, we develop three grading schemes that provide incentives for truthful peer grading. In the first scheme, the instructor grades a fraction p of the submissions, and penalizes students whose grade deviates from the instructor grade. We provide lower bounds on p to ensure truthfulness, and conclude that these schemes work only for moderate class sizes, up to a few hundred students. To overcome this limitation, we propose a hierarchical extension of this supervised scheme, and we show that it can handle classes of any size with bounded (and little) instructor work, and is therefore applicable to Massive Open Online Courses (MOOCs). Finally, we propose unsupervised incentive schemes, in which the student incentive is based on statistical properties of the grade distribution, without any grading required by the instructor. We show that the proposed unsupervised schemes provide incentives to truthful grading, at the price of being possibly unfair to individual students.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑