保持社区在循环中:理解基于机器学习系统的维基百科利益相关者价值
Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
摘要:
在维基百科上,复杂的算法工具被用来评估编辑的质量并采取纠正措施。然而,如果算法与使用它们的社区的价值观相冲突,它们可能无法解决它们设计的问题。在这项研究中,我们采取了价值敏感的算法设计方法来理解社区创建和维护的基于机器学习的算法,称为客观修订评估系统(ORES)-一个质量预测系统,用于许多维基百科的应用程序和上下文。ORES(及其依赖的应用程序)应该在利益相关者群体中汇聚五个主要价值观:(1)减少社区维护的工作,(2)保持人类判断作为最终权威,(3)支持不同人的不同工作流程,(4)鼓励与不同的编辑团体积极参与,以及(5)在社区内建立人和算法的可信度。我们揭示了这些值之间的紧张关系,并讨论了未来的研究,以改善算法,如ORES的影响。
英文摘要:
On Wikipedia, sophisticated algorithmic tools are used to assess the quality of edits and take corrective actions. However, algorithms can fail to solve the problems they were designed for if they conflict with the values of communities who use them. In this study, we take a Value-Sensitive Algorithm Design approach to understanding a community-created and -maintained machine learning-based algorithm called the Objective Revision Evaluation System (ORES)---a quality prediction system used in numerous Wikipedia applications and contexts. Five major values converged across stakeholder groups that ORES (and its dependent applications) should: (1) reduce the effort of community maintenance, (2) maintain human judgement as the final authority, (3) support differing peoples' differing workflows, (4) encourage positive engagement with diverse editor groups, and (5) establish trustworthiness of people and algorithms within the community. We reveal tensions between these values and discuss implications for future research to improve algorithms like ORES.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。