立场：想要更好的机器学习评审？别再客气请求，改用积分体系来激励
Position: Want Better ML Reviews? Stop Asking Nicely and Start Incentivizing with a Credit System
浏览论文内容
中文总结 AI 辅助
本文针对机器学习同行评审存在的投稿量难控、评审激励不足问题，提出需结合可执行的精细程序保障与OpenReview积分体系，以改进现有评审机制。
中文摘要 AI 辅助
随着投稿数量激增、 reciprocal评审政策愈发严格、OpenReview等平台被广泛采用，且没有发表费用的压力抵消，机器学习（ML）社区在所有科学领域中拥有最大的学术影响力之一。然而，几乎每个人都对自己的评审体验有诸多不满，更糟糕的是，几乎没有公开空间来认真讨论，更不用说辩论是什么让评审系统有效，或者它可能如何改进。在这篇立场论文中，我们从两个核心问题展开讨论：如何合理限制投稿量？以及如何激励良好的评审并阻止不良评审？我们首先评估现有解决此类问题的尝试的优缺点，具体而言，我们对一些流行的会议机制提出四种看法，并提出两种改进的替代设计。我们的总体立场是，机器学习同行评审的有意义改进不会来自嵌入在征文通知或评审指南中的礼貌最佳实践建议：它需要可执行且精细的程序保障，以及类似货币的积分体系（例如，我们提出的OpenReview积分）。ML从业者可以通过践行良好评审实践“赚取”此类积分，并在一个或多个主要会议中“消费”它们，以兑换不同类型的“福利”，例如免费注册或请求额外评审资源的权利。
英文摘要
With soaring submission counts, stricter reciprocal review policies, widespread adoption of platforms like OpenReview, and without the offsetting pressure of publication fees, the machine learning (ML) community has one of the largest scholarly presences among all scientific fields. And yet, \textbf{almost \textit{everyone} has \textit{many} unpleasant things to share about their review experience.} Worse, there is little public space to seriously discuss, let alone debate, what makes a review system effective or how it might be improved.\quad In this position paper, we expand our discussion from two core problems: \textit{How can we reasonably limit submission volume?} and \textit{How can we incentivize good and discourage bad reviewing?} We first assess the strengths and shortcomings of existing attempts to address such problems. Specifically, we present four takes on some popular conference mechanisms and propose two alternative designs for improvement.\quad Our general position is that meaningful improvement in ML peer review won't come from polite best-practice suggestions tucked into Calls for Papers or Reviewer Guidelines: it requires \textbf{enforceable yet fine-grained procedural safeguards} paired with \textbf{a currency-like credit system (e.g., our proposed \textit{OpenReview Points})}. ML practitioners can ``earn'' such points by contributing good review practices, and ``spend'' them across one or multiple major conferences to redeem different kinds of ``perks,'' such as complimentary registration or the right to request additional review resources.
