“直到投入生产，我们才知道模型在生产环境中会如何表现”：工程师如何实现机器学习的运营化
"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
AI总结：
通过对18名机器学习工程师的访谈，研究揭示了包含数据准备、实验、多阶段部署评估和持续监控的MLOps工作流，并提出速度、可见性和版本控制三大成功部署要素。
摘要：
组织依赖机器学习工程师将模型部署到生产环境并维护机器学习流水线。由于模型严重依赖新鲜数据，机器学习的运营化（即MLOps）要求工程师精通数据科学与工程。从整体来看，这项工作似乎极其繁重——工程师如何开展MLOps？他们面临哪些未解决的挑战？为解答这些问题，我们对18名从事聊天机器人、自动驾驶汽车和金融等应用的工程师进行了半结构化民族志访谈。我们发现，工程师的工作流包括：数据准备、实验、在多阶段部署中进行评估，以及持续监控与响应。在此流程中，工程师与数据科学家、产品利益相关者及彼此广泛协作，除了常规口头交流，还使用从Slack到全组织范围的工单与报告系统等通信工具作为补充。我们提出了MLOps的3V特性：速度、可见性和版本控制——这是成功部署机器学习的三大要素，工程师随着经验成熟会学会平衡并发展这些要素。最后，我们讨论了设计启示与未来工作的机遇。
英文摘要：
Organizations rely on machine learning engineers (MLEs) to deploy models and maintain ML pipelines in production. Due to models' extensive reliance on fresh data, the operationalization of machine learning, or MLOps, requires MLEs to have proficiency in data science and engineering. When considered holistically, the job seems staggering -- how do MLEs do MLOps, and what are their unaddressed challenges? To address these questions, we conducted semi-structured ethnographic interviews with 18 MLEs working on various applications, including chatbots, autonomous vehicles, and finance. We find that MLEs engage in a workflow of (i) data preparation, (ii) experimentation, (iii) evaluation throughout a multi-staged deployment, and (iv) continual monitoring and response. Throughout this workflow, MLEs collaborate extensively with data scientists, product stakeholders, and one another, supplementing routine verbal exchanges with communication tools ranging from Slack to organization-wide ticketing and reporting systems. We introduce the 3Vs of MLOps: velocity, visibility, and versioning -- three virtues of successful ML deployments that MLEs learn to balance and grow as they mature. Finally, we discuss design implications and opportunities for future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
