制造智能：IQ与ML基准中的伦理价值
Making Intelligence: Ethical Values in IQ and ML Benchmarks
AI总结：
本文指出ML基准与人类智能基准在结构上相似且均承载伦理价值，借鉴女性主义科学哲学论证基准创建需考虑并记录价值，并提出研究伦理建议。
摘要：
近年来，机器学习（ML）研究者一直在努力定义和改进机器学习基准与数据集。与此同时，一些人开始以批判性的视角审视数据集创建和机器学习研究的伦理问题。在这篇立场论文中，我们强调了伦理与看似“技术性”或“科学性”的ML基准设计决策之间的纠缠关系。我们的出发点是人类智能基准与机器学习基准之间存在多个被忽视的结构相似性。这两类基准都为描述、评估和比较与智能相关的任务表现设定了标准——而许多研究人类智能的学者早已认识到这些标准承载着价值。我们借鉴科学哲学中女性主义视角对IQ基准的分析以及社会科学中的厚概念，论证在创建ML基准时需要考虑并记录价值。创建价值中立的基准既不可能也不可取，无法回避这一选择。最后，我们为ML基准研究伦理和伦理审查提出了实用建议。
英文摘要：
In recent years, ML researchers have wrestled with defining and improving machine learning (ML) benchmarks and datasets. In parallel, some have trained a critical lens on the ethics of dataset creation and ML research. In this position paper, we highlight the entanglement of ethics with seemingly ``technical'' or ``scientific'' decisions about the design of ML benchmarks. Our starting point is the existence of multiple overlooked structural similarities between human intelligence benchmarks and ML benchmarks. Both types of benchmarks set standards for describing, evaluating, and comparing performance on tasks relevant to intelligence -- standards that many scholars of human intelligence have long recognized as value-laden. We use perspectives from feminist philosophy of science on IQ benchmarks and thick concepts in social science to argue that values need to be considered and documented when creating ML benchmarks. It is neither possible nor desirable to avoid this choice by creating value-neutral benchmarks. Finally, we outline practical recommendations for ML benchmark research ethics and ethics review.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
