内容审核中的算法任意性
Algorithmic Arbitrariness in Content Moderation
AI总结：
本文研究最先进LLM有毒内容检测中的预测多样性及其群体差异和与人工分类的差异，揭示算法审核对言论自由等人权的任意性风险，并呼吁提升透明度以支撑相关政策与立法。
摘要：
机器学习（ML）被广泛用于审核在线内容。尽管其相对于人工审核具有可扩展性，但使用ML也给内容审核带来了独特挑战。其中一个挑战是预测多样性：多个相互竞争的内容分类模型在平均表现上可能同样出色，却会对同一内容给出相互冲突的预测。这种多样性可能源于模型开发过程中看似无害的选择，例如参数初始化的随机种子选择。我们通过实验展示了内容审核工具如何任意地将样本分类为有毒，从而导致对言论的任意限制。我们结合《公民权利和政治权利国际公约》（ICCPR）所规定的人权，即表达自由、非歧视和程序正义，讨论了这些发现。我们分析了：（i）用于检测有毒内容的最先进LLM之间预测多样性的程度；（ii）这种任意性在不同社会群体间造成的差异性影响；以及（iii）模型多样性与明确的人类分类相比如何。研究结果表明，规模化的算法审核有可能使一个算法“利维坦”合法化，即算法不成比例地管理人 权。为降低此类风险，本研究强调有必要识别并提高内容审核应用中任意性的透明度。由于算法内容审核正受到虚假信息和仇恨言论等紧迫社会关切的推动，我们关于危害的讨论提出了与政策辩论相关的担忧。我们的发现也有助于许多国家正在讨论和通过的内容审核及中介责任法律，例如欧盟的《数字服务法》、英国的《在线安全法》以及巴西的《假新闻法案》。
英文摘要：
Machine learning (ML) is widely used to moderate online content. Despite its scalability relative to human moderation, the use of ML introduces unique challenges to content moderation. One such challenge is predictive multiplicity: multiple competing models for content classification may perform equally well on average, yet assign conflicting predictions to the same content. This multiplicity can result from seemingly innocuous choices during model development, such as random seed selection for parameter initialization. We experimentally demonstrate how content moderation tools can arbitrarily classify samples as toxic, leading to arbitrary restrictions on speech. We discuss these findings in terms of human rights set out by the International Covenant on Civil and Political Rights (ICCPR), namely freedom of expression, non-discrimination, and procedural justice. We analyze (i) the extent of predictive multiplicity among state-of-the-art LLMs used for detecting toxic content; (ii) the disparate impact of this arbitrariness across social groups; and (iii) how model multiplicity compares to unambiguous human classifications. Our findings indicate that the up-scaled algorithmic moderation risks legitimizing an algorithmic leviathan, where an algorithm disproportionately manages human rights. To mitigate such risks, our study underscores the need to identify and increase the transparency of arbitrariness in content moderation applications. Since algorithmic content moderation is being fueled by pressing social concerns, such as disinformation and hate speech, our discussion on harms raises concerns relevant to policy debates. Our findings also contribute to content moderation and intermediary liability laws being discussed and passed in many countries, such as the Digital Services Act in the European Union, the Online Safety Act in the United Kingdom, and the Fake News Bill in Brazil.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
