arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.05862cs.LGcs.AIcs.CY
收藏

对已部署机器学习的生态系统层面分析揭示同质化结果

Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomes

Stanford University(斯坦福大学);Northeastern University(东北大学);Chapman University(查普曼大学)

Connor Toups, Rishi Bommasani, Kathleen A. Creel, Sarah H. Bana, Dan Jurafsky, Percy Liang

AI总结:

本文提出生态系统层面的机器学习分析方法,通过三种模态、11个数据集及皮肤科影像案例,揭示已部署模型会导致同质化的系统性失败和新型种族差异。

摘要:

传统上,机器学习是在模型层面被研究的:研究人员测量并改进特定模型的准确性、鲁棒性、偏见、效率及其他维度。在实践中,机器学习的社会影响取决于机器学习部署所处的周边环境。为捕捉这一点,我们引入生态系统层面分析:我们不分析单个模型,而是考察在给定环境中部署的模型集合。例如,招聘中的生态系统层面分析认识到,求职者的结果不仅由单一招聘算法或公司决定,而是由其申请的所有公司的集体决策决定。在文本、图像、语音三种模态以及11个数据集上,我们确立了一个明显趋势:已部署的机器学习容易发生系统性失败,这意味着一些用户会被所有可用模型无一例外地错误分类。即使单个模型随时间在总体层面有所改进,我们发现这些改进也很少降低系统性失败的发生率。相反,这些改进带来的收益主要流向那些已经被其他模型正确分类的个体。鉴于这些趋势,我们考察了系统性失败代价尤其高昂的皮肤科医学影像。尽管传统分析揭示了模型和人类在表现上都存在种族差异,但生态系统层面分析揭示了模型预测中存在的、人类预测中并未出现的新型种族差异。这些例子表明,生态系统层面分析在刻画机器学习的社会影响方面具有独特优势。

英文摘要:

Machine learning is traditionally studied at the model level: researchers measure and improve the accuracy, robustness, bias, efficiency, and other dimensions of specific models. In practice, the societal impact of machine learning is determined by the surrounding context of machine learning deployments. To capture this, we introduce ecosystem-level analysis: rather than analyzing a single model, we consider the collection of models that are deployed in a given context. For example, ecosystem-level analysis in hiring recognizes that a job candidate's outcomes are not only determined by a single hiring algorithm or firm but instead by the collective decisions of all the firms they applied to. Across three modalities (text, images, speech) and 11 datasets, we establish a clear trend: deployed machine learning is prone to systemic failure, meaning some users are exclusively misclassified by all models available. Even when individual models improve at the population level over time, we find these improvements rarely reduce the prevalence of systemic failure. Instead, the benefits of these improvements predominantly accrue to individuals who are already correctly classified by other models. In light of these trends, we consider medical imaging for dermatology where the costs of systemic failure are especially high. While traditional analyses reveal racial performance disparities for both models and humans, ecosystem-level analysis reveals new forms of racial disparity in model predictions that do not present in human predictions. These examples demonstrate ecosystem-level analysis has unique strengths for characterizing the societal impact of machine learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑