“它出现在笔记本中”：机器学习原型投入运营时的变化与挑战
"It Comes in Notebooks": Changes and Challenges when Operationalizing ML Prototypes
- Delft University of Technology（代尔夫特理工大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究通过访谈13位机器学习从业者，识别出原型到生产过渡中的23项工程变更和20个质量属性，提出“监督债务”概念，并揭示七个质量权衡，为运营化实践提供结构化指导。
AI中文摘要：
机器学习从业者通常先在计算笔记本中构建原型，然后再将其过渡到自动化生产系统。尽管这种做法很普遍，但这一过渡所涉及的具体工程工作及其引发的软件质量关注仍未得到充分描述。我们报告了一项基于对来自工业界和学术界的13位机器学习从业者进行半结构化访谈的定性研究。通过反思性主题分析，我们识别出23项工程变更，并将其归纳为五个主题：代码重构、数据管道开发、测试与验证、管道自动化以及监控与可观测性。我们还识别出机器学习开发生命周期中的20个软件质量属性，并将其映射到这些工程变更上。我们研究中的一个反复出现的模式是，计算笔记本将监督外部化给人类从业者，并将成本推迟到运营化时期才变得不可避免。运营化构成了对原型阶段积累的技术债务的偿还，我们将其称为监督债务。从业者不仅仅是重构笔记本代码，而是通过构建自动化替代品来偿还这种债务，以替代笔记本所提供的交互式监督。我们进一步提出了七个质量权衡，表明这些张力是笔记本到生产过渡的属性，而非不良工程实践的征兆。我们的研究结果构建了运营化工作的结构，建立了工程变更与软件质量关注之间的实证联系，并为从事机器学习驱动软件系统的从业者、工具设计者和研究人员提供了启示。
英文摘要：
Machine learning practitioners commonly prototype models in computational notebooks before transitioning them to automated production systems. Despite its prevalence, the concrete engineering work involved in this transition and the software quality concerns that motivate it remain insufficiently characterized. We report on a qualitative study based on semi-structured interviews with 13 ML practitioners from industry and academia. Using reflexive thematic analysis, we identify 23 engineering changes organized into five themes: code restructuring, data pipeline development, testing & validation, pipeline automation, and monitoring & observability. We also identify 20 software quality attributes across the ML development lifecycle and map them to the engineering changes. A recurring pattern in our findings is that computational notebooks externalize oversight to the human practitioner, and defer costs that become obligatory at operationalization time. Operationalization constitutes the repayment of this technical debt accumulated during prototyping, which we refer to as oversight debt. Practitioners do not merely restructure notebook code, but repay this debt by constructing automated substitutes for the interactive oversight that notebooks provide. We further present seven quality trade-offs showing that these tensions are properties of the notebook-to-production transition, rather than symptoms of poor engineering practice. Our findings structure operationalization effort, establish empirical links between engineering changes and software quality concerns, and provide implications for practitioners, tool designers, and researchers working on ML-enabled software systems.
