《Catch Me If I Can：Detecting Strategy Behavior in Peer Assessment》（如果我能抓住我：在同伴评估中发现战略行为）
Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment
摘要：
我们认为，在各种同行评估任务，包括考试或作业的同行评分和同行评审在招聘或晋升的战略行为的问题。当同行评估任务具有竞争性时（例如，当学生在曲线上评分时），代理人可能会被激励误报评估，以提高他们自己的最终排名。我们的重点是设计检测这种操作的方法。具体来说，我们考虑一个设置中，代理评估他们的同行和输出排名，后来汇总，形成一个最终的排序的子集。在本文中，我们调查了这个问题的统计框架，并设计了一个原则性的测试检测战略行为。我们证明了我们的测试有很强的虚警保证，并评估其在实际环境中的检测能力。为此，我们设计并执行了一个实验，从受试者中提取战略行为，并发布了一个可能具有独立兴趣的战略行为模式数据集。然后，我们使用收集的数据进行一系列真实和半合成的评估，证明了我们的测试的强大检测能力。
英文摘要：
We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions. When a peer-assessment task is competitive (e.g., when students are graded on a curve), agents may be incentivized to misreport evaluations in order to improve their own final standing. Our focus is on designing methods for detection of such manipulations. Specifically, we consider a setting in which agents evaluate a subset of their peers and output rankings that are later aggregated to form a final ordering. In this paper, we investigate a statistical framework for this problem and design a principled test for detecting strategic behaviour. We prove that our test has strong false alarm guarantees and evaluate its detection ability in practical settings. For this, we design and execute an experiment that elicits strategic behaviour from subjects and release a dataset of patterns of strategic behaviour that may be of independent interest. We then use the collected data to conduct a series of real and semi-synthetic evaluations that demonstrate a strong detection power of our test.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
