我们如何定义“伤害”会影响数据标注：解释标注者如何区分仇恨、冒犯和有毒评论
How We Define Harm Impacts Data Annotations: Explaining How Annotators Distinguish Hateful, Offensive, and Toxic Comments
AI总结：
本研究通过维恩图、信息增益比较和内容分析，证明“仇恨”“冒犯”“有毒”等伤害概念不可互换，并建议研究人员依据目标明确界定概念、限定结论适用范围。
摘要：
计算社会科学研究在机器学习和自然语言处理方面取得了进展，这些进展支持内容审核人员检测有害内容。这些进展通常依赖于由众包工作者标注的有害内容训练数据集。在设计标注任务说明以为这些算法生成训练数据时，研究人员往往将我们训练算法去检测的伤害概念——“仇恨的”“冒犯的”“有毒的”“种族主义的”“性别歧视的”等——视为可以互换。在本研究中，我们考察了研究人员定义“伤害”的方式是否会影响标注结果。利用维恩图、信息增益比较和内容分析，我们揭示标注者并不会将“仇恨的”“冒犯的”和“有毒的”这些概念互换使用。我们发现，伤害定义的特征以及标注者的个人特征，能够在很大程度上解释标注者为何以不同方式使用这些术语。我们的结果提供了经验证据，不鼓励在内容审核研究中将伤害概念互换使用的常见做法。相反，研究人员应根据其研究目标，具体选择要分析哪些伤害概念。考虑到研究人员往往受到资源限制，我们还鼓励他们在自身关注的概念与现成有害内容检测算法所识别的概念不同时，提供信息以限定其研究发现的适用范围。最后，我们鼓励算法提供者确保其工具能够适应特定情境下的内容检测目标（例如，征求工具用户的反馈）。
英文摘要：
Computational social science research has made advances in machine learning and natural language processing that support content moderators in detecting harmful content. These advances often rely on training datasets annotated by crowdworkers for harmful content. In designing instructions for annotation tasks to generate training data for these algorithms, researchers often treat the harm concepts that we train algorithms to detect - 'hateful', 'offensive', 'toxic', 'racist', 'sexist', etc. - as interchangeable. In this work, we studied whether the way that researchers define 'harm' affects annotation outcomes. Using Venn diagrams, information gain comparisons, and content analyses, we reveal that annotators do not use the concepts 'hateful', 'offensive', and 'toxic' interchangeably. We identify that features of harm definitions and annotators' individual characteristics explain much of how annotators use these terms differently. Our results offer empirical evidence discouraging the common practice of using harm concepts interchangeably in content moderation research. Instead, researchers should make specific choices about which harm concepts to analyze based on their research goals. Recognizing that researchers are often resource constrained, we also encourage researchers to provide information to bound their findings when their concepts of interest differ from concepts that off-the-shelf harmful content detection algorithms identify. Finally, we encourage algorithm providers to ensure their instruments can adapt to contextually-specific content detection goals (e.g., soliciting instrument users' feedback).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
