生产机器学习管道：实证分析和优化机会
Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities
摘要：
机器学习（ML）现在已经很普遍，为各种组织中的数据驱动应用程序提供动力。与研究中对ML的传统看法不同，ML生产管道很复杂，除了训练之外还有许多互锁的分析组件，其子部分通常在重叠的数据子集上运行多次。然而，缺乏有关这些管道的寿命、架构、频率和复杂性的定量证据，无法了解如何使用数据管理研究来使它们更高效、更有效、更强大和更可复制。为此，我们分析了Google的3000个生产ML管道的起源图，其中包括超过450，000个经过训练的模型，时间跨度超过四个月，以了解生产ML的复杂性和挑战。我们的分析揭示了各种粒度下典型行业强度ML管道的特征，组件和拓扑结构。在此过程中，我们引入了一个专门的数据模型，用于表示和推理这些ML管道中重复运行的组件，我们称之为模型小图。我们确定了几个丰富的优化机会，利用传统的数据管理思想。我们展示了如何瞄准这些机会之一，即，识别和修剪没有转换为模型部署的浪费计算，可以将浪费的计算成本减少50%，而不会影响模型部署节奏。
英文摘要：
Machine learning (ML) is now commonplace, powering data-driven applications in various organizations. Unlike the traditional perception of ML in research, ML production pipelines are complex, with many interlocking analytical components beyond training, whose sub-parts are often run multiple times on overlapping subsets of data. However, there is a lack of quantitative evidence regarding the lifespan, architecture, frequency, and complexity of these pipelines to understand how data management research can be used to make them more efficient, effective, robust, and reproducible. To that end, we analyze the provenance graphs of 3000 production ML pipelines at Google, comprising over 450,000 models trained, spanning a period of over four months, in an effort to understand the complexity and challenges underlying production ML. Our analysis reveals the characteristics, components, and topologies of typical industry-strength ML pipelines at various granularities. Along the way, we introduce a specialized data model for representing and reasoning about repeatedly run components in these ML pipelines, which we call model graphlets. We identify several rich opportunities for optimization, leveraging traditional data management ideas. We show how targeting even one of these opportunities, i.e., identifying and pruning wasted computation that does not translate to model deployment, can reduce wasted computation cost by 50% without compromising the model deployment cadence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
