利用分布不等式度量内容推荐算法的不同结果
Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics
摘要：
算法决策系统的有害影响最近成为焦点，许多系统的例子，如机器学习（ML）模型放大了现有的社会偏见。大多数试图量化ML算法产生的差异的指标都集中在组之间的差异上，根据人口统计特征划分用户，并比较这些组之间的模型性能或总体结果。然而，在行业环境中，这些信息往往是不可用的，推断这些特征有其自身的风险和偏见。此外，关注单个分类器输出的典型指标忽略了在现实世界中产生结果的复杂系统网络。在本文中，我们评估了一组来自经济学的指标，分布不平等的指标，以及他们的能力来衡量生产推荐系统，Twitter的算法时间轴的内容曝光的差距。我们为在操作环境中使用的指标定义了理想的标准，特别是ML从业者。我们使用这些指标来描述Twitter上不同类型的内容参与，并使用这些结果来评估所需标准的指标。我们表明，我们可以使用这些指标来确定内容建议算法，有助于更强烈地倾斜用户之间的结果。总体而言，我们的结论是，这些指标可以理解在网上社交网络不同的结果是有用的工具。
英文摘要：
The harmful impacts of algorithmic decision systems have recently come into focus, with many examples of systems such as machine learning (ML) models amplifying existing societal biases. Most metrics attempting to quantify disparities resulting from ML algorithms focus on differences between groups, dividing users based on demographic identities and comparing model performance or overall outcomes between these groups. However, in industry settings, such information is often not available, and inferring these characteristics carries its own risks and biases. Moreover, typical metrics that focus on a single classifier's output ignore the complex network of systems that produce outcomes in real-world settings. In this paper, we evaluate a set of metrics originating from economics, distributional inequality metrics, and their ability to measure disparities in content exposure in a production recommendation system, the Twitter algorithmic timeline. We define desirable criteria for metrics to be used in an operational setting, specifically by ML practitioners. We characterize different types of engagement with content on Twitter using these metrics, and use these results to evaluate the metrics with respect to the desired criteria. We show that we can use these metrics to identify content suggestion algorithms that contribute more strongly to skewed outcomes between users. Overall, we conclude that these metrics can be useful tools for understanding disparate outcomes in online social networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
