同行评议中论文竞价优化的SUPER * 算法
A SUPER* Algorithm to Optimize Paper Bidding in Peer Review
摘要:
许多应用程序涉及用户的顺序到达,并且需要向每个用户显示物品的顺序。一个主要的例子(形成本文的重点)是投标过程中的会议同行评审员进入系统顺序,每个评审员需要提交的论文列表,然后评审员“投标”审查一些论文。由于首因效应,论文的顺序对出价有很大的影响。在决定论文的展示顺序时,有两个相互竞争的目标:(i)为每篇论文获得足够多的出价,(ii)通过向评审员展示相关项目来满足他们。在本文中,我们首先开发了一个框架,以原则性的方式研究这个问题。我们提出了一种算法,称为超级 *,灵感来自A* 算法,为这个目标。理论上,我们展示了我们的算法的局部最优性保证,并证明流行的基线是相当次优的。此外,根据社区模型的相似性,我们证明了超级 * 是接近最优的,而流行的基线是相当次优的。在对ICLR 2018和合成数据的真实数据进行的实验中,我们发现SUPER* 的性能大大优于现有系统中部署的基线,持续减少了50-75%或更多的投标论文数量,并且对各种现实世界的复杂性也很强大。
英文摘要:
A number of applications involve sequential arrival of users, and require showing each user an ordering of items. A prime example (which forms the focus of this paper) is the bidding process in conference peer review where reviewers enter the system sequentially, each reviewer needs to be shown the list of submitted papers, and the reviewer then "bids" to review some papers. The order of the papers shown has a significant impact on the bids due to primacy effects. In deciding on the ordering of papers to show, there are two competing goals: (i) obtaining sufficiently many bids for each paper, and (ii) satisfying reviewers by showing them relevant items. In this paper, we begin by developing a framework to study this problem in a principled manner. We present an algorithm called SUPER*, inspired by the A* algorithm, for this goal. Theoretically, we show a local optimality guarantee of our algorithm and prove that popular baselines are considerably suboptimal. Moreover, under a community model for the similarities, we prove that SUPER* is near-optimal whereas the popular baselines are considerably suboptimal. In experiments on real data from ICLR 2018 and synthetic data, we find that SUPER* considerably outperforms baselines deployed in existing systems, consistently reducing the number of papers with fewer than requisite bids by 50-75% or more, and is also robust to various real world complexities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。