Recent successes in the Machine Learning community have led to a steep increase in the number of papers submitted to conferences. This increase made more prominent some of the issues that affect the current review process used by these conferences. The review process has several issues that may undermine the nature of scientific research, which is of being fully objective, apolitical, unbiased and free of misconduct (such as plagiarism, cheating, improper influence, and other improprieties). In this work, we study the problem of reviewers' recruitment, infringements of the double-blind process, fraudulent behaviors, biases in numerical ratings, and the appendix phenomenon (i.e., the fact that it is becoming more common to publish results in the appendix section of a paper). For each of these problems, we provide a short description and possible solutions. The goal of this work is to raise awareness in the Machine Learning community regarding these issues.
Analyzing the Machine Learning Conference Review Process
University of California, Berkeley(加州大学伯克利分校);Princeton University(普林斯顿大学);Centennial High School(百年高中);Harvard University(哈佛大学);University of Maryland, College Park(马里兰大学帕克分校)
An Open Review of OpenReview: A Critical Analysis of the Machine Learning Conference Review Process
University of California, Berkeley(加州大学伯克利分校);Princeton University(普林斯顿大学);Centennial High School(百年高中);Harvard University(哈佛大学);University of Maryland, College Park(马里兰大学帕克分校);University of Maryland College Park(马里兰大学帕克分校)
Design and Analysis of the NIPS 2016 Review Process
Carnegie Mellon University(卡内基梅隆大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所);Universite Paris-Saclay(巴黎萨克雷大学);ChaLearn(ChaLearn(查尔恩))
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
Department of Computer Science, Stanford University(斯坦福大学计算机科学系);Machine Learning Department, NEC Labs America(NEC美国实验室机器学习部门);Department of Biomedical Data Science, Stanford University(斯坦福大学生物医学数据科学系);Department of Electrical Engineering, Stanford University(斯坦福大学电气工程系);Graduate School of Education, Stanford University(斯坦福大学教育研究生院);Department of Sociology, Stanford University(斯坦福大学社会学系);Graduate School of Business, Stanford University(斯坦福大学商学院);Department of Management Science and Engineering, Stanford University(斯坦福大学管理科学与工程系);Department of Computer Science, UC Santa Barbara(加州大学圣芭芭拉分校计算机科学系)
Generative Artificial Intelligence in Healthcare: Ethical Considerations and Assessment Checklist
Duke-NUS Medical School(杜克-新加坡国立大学医学院);National University of Singapore(新加坡国立大学);University of Oxford(牛津大学);Imperial College London(帝国理工学院);University of Antwerp(安特卫普大学);Singapore Eye Research Institute(新加坡眼科研究所);Singapore National Eye Centre(新加坡全国眼科中心);Singapore Health Services(新加坡保健服务);Singapore General Hospital(新加坡中央医院);Hangzhou Dianzi University(杭州电子科技大学);Nanyang Technological University(南洋理工大学);The Chinese University of Hong Kong(香港中文大学);Scripps Research(斯克里普斯研究所)
No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning
Ruhr University Bochum(波鸿鲁尔大学);International Computer Science Institute (ICSI)(国际计算机科学研究所);Technische Universität Berlin(柏林工业大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心)