arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.07161cs.LGcs.SEstat.ML
收藏

Collective Knowledge项目:通过开放API、可重用最佳实践和MLOp,使ML模型更具可移植性和可复制性

The Collective Knowledge project: making ML models more portable and reproducible with open APIs, reusable best practices and MLOps

cTuning foundation(cTuning基金会);cKnowledge SAS(cKnowledge SAS公司)

Grigori Fursin

摘要:

本文概述了集体知识技术(CK或cKnowledge)。CK试图让机器学习和系统研究更容易重现,在生产中部署机器学习模型,并使其适应不断变化的数据集、模型、研究技术、软件和硬件。CK的概念是将复杂的系统和特别的研究项目分解为具有统一API、CLI和JSON Meta描述的可重用子组件。这些组件可以使用DevOps原则与可重用的自动化操作、软件检测插件、Meta包和公开的优化参数相结合连接到可移植的工作流中。CK工作流可以自动插件来自不同供应商的不同模型、数据和工具,同时在不同的平台和环境中以统一的方式构建、运行和基准测试研究代码。这样的工作流程还有助于执行整个系统优化,重现结果,并使用CK平台上的公共或私人记分牌(https://www.example.com)进行比较。cKnowledge.io例如,模块化CK方法已与工业合作伙伴成功验证,可自动协同设计和优化软件、硬件和机器学习模型,以实现速度、准确性、能量、尺寸和其他特性方面的可再现和高效目标检测。长期目标是通过帮助研究人员和从业者使用开放的CK API共享和重用他们的知识、经验、最佳实践、工件和技术,简化和加速ML模型和系统的开发和部署。

英文摘要:

This article provides an overview of the Collective Knowledge technology (CK or cKnowledge). CK attempts to make it easier to reproduce ML&systems research, deploy ML models in production, and adapt them to continuously changing data sets, models, research techniques, software, and hardware. The CK concept is to decompose complex systems and ad-hoc research projects into reusable sub-components with unified APIs, CLI, and JSON meta description. Such components can be connected into portable workflows using DevOps principles combined with reusable automation actions, software detection plugins, meta packages, and exposed optimization parameters. CK workflows can automatically plug in different models, data and tools from different vendors while building, running and benchmarking research code in a unified way across diverse platforms and environments. Such workflows also help to perform whole system optimization, reproduce results, and compare them using public or private scoreboards on the CK platform (https://cKnowledge.io). For example, the modular CK approach was successfully validated with industrial partners to automatically co-design and optimize software, hardware, and machine learning models for reproducible and efficient object detection in terms of speed, accuracy, energy, size, and other characteristics. The long-term goal is to simplify and accelerate the development and deployment of ML models and systems by helping researchers and practitioners to share and reuse their knowledge, experience, best practices, artifacts, and techniques using open CK APIs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑