arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.10312cs.CYcs.HC
收藏

超越机器学习模型:将安全工程框架应用于文本生成图像开发

Beyond the ML Model: Applying Safety Engineering Frameworks to Text-to-Image Development

Shalaleh Rismani, Renee Shelby, Andrew Smart, Renelito Delos Santos, AJung Moon, Negar Rostamzadeh

AI总结:

本研究将FMEA、STPA两种安全工程框架应用于文本生成图像模型开发全流程,证明其可发掘多层面的社会伦理风险,凸显了跳出模型本身开展风险审查的价值。

摘要:

在新兴机器学习(ML)模型及其应用中识别潜在的社会与伦理风险仍是一项挑战。本研究将两种成熟的安全工程框架——失效模式与影响分析(FMEA)、系统理论过程分析(STPA)应用于一项涉及文本生成图像(T2I)模型的案例研究,覆盖ML产品开发管线的三个阶段:数据处理、T2I模型与其他模型的集成、使用阶段。分析结果表明,这两种并非专门用于审查社会与伦理风险的安全框架,能够揭示构成社会与伦理风险的失效与危害。我们通过分析交互关系(即产品内不同ML模型间的交互、ML产品与用户的交互、开发团队间的交互)与流程(即训练数据准备或ML服务/产品的使用工作流),发现了涵盖功能、社会与伦理层面的广泛失效与危害。研究结论强调,在审查社会与伦理风险时,突破ML模型本身的局限开展审查极具价值与重要性,尤其是在我们对ML模型掌握信息极少的情况下。

英文摘要:

Identifying potential social and ethical risks in emerging machine learning (ML) models and their applications remains challenging. In this work, we applied two well-established safety engineering frameworks (FMEA, STPA) to a case study involving text-to-image models at three stages of the ML product development pipeline: data processing, integration of a T2I model with other models, and use. Results of our analysis demonstrate the safety frameworks - both of which are not designed explicitly examine social and ethical risks - can uncover failure and hazards that pose social and ethical risks. We discovered a broad range of failures and hazards (i.e., functional, social, and ethical) by analyzing interactions (i.e., between different ML models in the product, between the ML product and user, and between development teams) and processes (i.e., preparation of training data or workflows for using an ML service/product). Our findings underscore the value and importance of examining beyond an ML model in examining social and ethical risks, especially when we have minimal information about an ML model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑