同行评审中的群体公平性
Group Fairness in Peer Review
AI总结:
针对大型会议中部分社群投稿评审分配不公的问题,提出名为core的群体公平性概念,证明同行评审模型存在满足该公平性的分配方案,设计高效算法并通过真实会议数据验证其效果。
摘要:
NeurIPS、AAAI等大型会议是人工智能各领域的交汇点,吸引了大量研究社群的投稿。但在部分情况下,这会导致部分社群获得糟糕的评审体验——它们的投稿被分配给社群外资质不足的评审员。人们常提倡的解决方案是将这类大型会议拆分为更小的会议,但这会造成社群孤立,损害跨学科研究。我们通过引入名为core(核心)的群体公平性概念来应对这一挑战,该概念要求所有可能的社群(研究者子集)都得到公平对待,使其无法通过退出大型会议单方面获益。我们研究了一个简单的同行评审模型,证明该模型始终存在属于core的评审分配方案,并设计了一种高效算法来找到这类分配。我们使用CVPR和ICLR会议的真实数据,在多项指标上将我们的算法与现有评审分配算法进行了对比。
英文摘要:
Large conferences such as NeurIPS and AAAI serve as crossroads of various AI fields, since they attract submissions from a vast number of communities. However, in some cases, this has resulted in a poor reviewing experience for some communities, whose submissions get assigned to less qualified reviewers outside of their communities. An often-advocated solution is to break up any such large conference into smaller conferences, but this can lead to isolation of communities and harm interdisciplinary research. We tackle this challenge by introducing a notion of group fairness, called the core, which requires that every possible community (subset of researchers) to be treated in a way that prevents them from unilaterally benefiting by withdrawing from a large conference. We study a simple peer review model, prove that it always admits a reviewing assignment in the core, and design an efficient algorithm to find one such assignment. We use real data from CVPR and ICLR conferences to compare our algorithm to existing reviewing assignment algorithms on a number of metrics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。