Modyn:以数据为中心的机器学习管道编排
Modyn: Data-Centric Machine Learning Pipeline Orchestration
AI总结:
Modyn是一个以数据为中心的端到端机器学习平台,通过声明式管道抽象支持持续训练,引入复合模型概念衡量性能,并实证分析数据选择与触发策略对准确率的影响。
摘要:
在现实世界的机器学习(ML)管道中,数据集持续增长。模型必须纳入这些新的训练数据,以提升泛化能力并适应潜在的分布偏移。模型再训练的成本与再训练的频率以及训练所用的数据量成正比,这使得每次从头再训练的朴素方法变得不切实际。我们提出了 Modyn,一个以数据为中心的端到端机器学习平台。Modyn 的 ML 管道抽象使用户能够以声明式方式描述在增长的数据集上持续训练模型的策略。Modyn 管道允许用户应用数据选择策略(以减少数据点数量)和触发策略(以减少训练次数)。Modyn 执行并编排这些持续性的 ML 训练管道。该系统是开源的,并附带一个包含基准数据集、模型和工具的生态系统。我们通过引入复合模型(composite models)的概念,正式讨论了如何衡量 ML 管道的性能,从而实现对具有不同数据选择和触发策略的管道进行公平比较。我们实证分析了各种数据选择和触发策略对模型准确率的影响,并展示了 Modyn 能够支持在样本级数据选择下实现高吞吐量的训练。
英文摘要:
In real-world machine learning (ML) pipelines, datasets are continuously growing. Models must incorporate this new training data to improve generalization and adapt to potential distribution shifts. The cost of model retraining is proportional to how frequently the model is retrained and how much data it is trained on, which makes the naive approach of retraining from scratch each time impractical. We present Modyn, a data-centric end-to-end machine learning platform. Modyn's ML pipeline abstraction enables users to declaratively describe policies for continuously training a model on a growing dataset. Modyn pipelines allow users to apply data selection policies (to reduce the number of data points) and triggering policies (to reduce the number of trainings). Modyn executes and orchestrates these continuous ML training pipelines. The system is open-source and comes with an ecosystem of benchmark datasets, models, and tooling. We formally discuss how to measure the performance of ML pipelines by introducing the concept of composite models, enabling fair comparison of pipelines with different data selection and triggering policies. We empirically analyze how various data selection and triggering policies impact model accuracy, and also show that Modyn enables high throughput training with sample-level data selection.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。