PeerReview4All:同行评议中评审员的公平准确分配
PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review
摘要:
我们认为,在会议同行评议的论文自动分配给审稿人的问题,重点是公平性和统计准确性。我们的公平性目标是最大限度地提高最弱势论文的评审质量,而通常使用的目标是最大限度地提高所有论文的总体质量。我们设计了一个分配算法的基础上,我们证明是近最佳公平的增量最大流过程。我们的统计准确性目标是确保正确恢复应被接受的论文。我们提供了一个尖锐的极小极大分析的同行评审过程中的一个流行的客观评分模型的准确性,以及一个新的主观评分模型,我们提出的文件。我们的分析证明,我们提出的分配算法也导致一个接近最佳的统计精度。最后,我们设计了一个新的实验,允许各种分配算法的客观比较,并克服了固有的困难所造成的同行评议实验中没有地面真理。本实验的结果以及其他实验的合成和真实数据证实了我们的算法的理论保证。
英文摘要:
We consider the problem of automated assignment of papers to reviewers in conference peer review, with a focus on fairness and statistical accuracy. Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers. We design an assignment algorithm based on an incremental max-flow procedure that we prove is near-optimally fair. Our statistical accuracy objective is to ensure correct recovery of the papers that should be accepted. We provide a sharp minimax analysis of the accuracy of the peer-review process for a popular objective-score model as well as for a novel subjective-score model that we propose in the paper. Our analysis proves that our proposed assignment algorithm also leads to a near-optimal statistical accuracy. Finally, we design a novel experiment that allows for an objective comparison of various assignment algorithms, and overcomes the inherent difficulty posed by the absence of a ground truth in experiments on peer-review. The results of this experiment as well as of other experiments on synthetic and real data corroborate the theoretical guarantees of our algorithm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。