社会伦理ML风险的系统安全工程:案例研究
System Safety Engineering for Social and Ethical ML Risks: A Case Study
摘要:
各国政府、工业界和学术界都在努力识别和减轻ML驱动系统的危害,特别关注ML组件在复杂社会技术系统中的社会和道德风险。然而,现有的方法在很大程度上是脱节的,临时的和未知的有效性。系统安全工程是一门成熟的学科,在许多复杂的社会技术领域都有识别和管理风险的记录。我们采用了自然的假设,即该领域的工具可以在其使用环境中增强ML的风险分析。为了验证这一假设,我们采用了“最佳品种”的系统安全性分析,系统理论过程分析(STPA),一个特定的高后果系统与一个重要的ML驱动的组件,即处方药监测计划(PDMP)由美国许多州,其中一些依赖于ML派生的风险评分。我们特别关注这种分析如何扩展到识别社会和道德风险,并制定具体的设计级控制措施来减轻这些风险。
英文摘要:
Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems, with a particular focus on social and ethical risks of ML components in complex sociotechnical systems. However, existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. Systems safety engineering is a well established discipline with a track record of identifying and managing risks in many complex sociotechnical domains. We adopt the natural hypothesis that tools from this domain could serve to enhance risk analyses of ML in its context of use. To test this hypothesis, we apply a "best of breed" systems safety analysis, Systems Theoretic Process Analysis (STPA), to a specific high-consequence system with an important ML-driven component, namely the Prescription Drug Monitoring Programs (PDMPs) operated by many US States, several of which rely on an ML-derived risk score. We focus in particular on how this analysis can extend to identifying social and ethical risks and developing concrete design-level controls to mitigate them.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。