集体知识:将研究项目组织为可重用组件和具有通用API的可移植工作流的数据库
Collective Knowledge: organizing research projects as a database of reusable components and portable workflows with common APIs
摘要:
本文提供了集体知识框架(CK或cKnowledge)的动机和概述。CK的概念是将研究项目分解为可重用的组件,这些组件封装了研究工件,并为相关工件提供统一的应用程序编程接口(API)、命令行接口(CLI)、Meta描述和公共自动化操作。CK框架用于组织和管理研究项目,作为这些组件的数据库。受USB“即插即用”硬件方法的启发,CK还帮助组装便携式工作流程,可以自动插入来自不同用户和供应商的兼容组件(模型、数据集、框架、编译器、工具)。此类工作流程可以使用带有软件检测插件的通用CK程序管道和自动安装丢失的包,以统一的方式在不同的平台和环境上构建和运行算法。本文介绍了一些工业项目,在这些项目中,模块化CK方法得到了成功验证,以便在速度,准确性,能源,大小和各种成本方面自动对机器学习(ML)和人工智能(AI)的高效软件和硬件进行基准测试,自动调整和协同设计。CK框架还有助于在几个计算机科学会议上自动化工件评估过程,并使其更容易复制,比较和重用已发表论文中的研究技术,将其部署在生产中,并自动使其适应不断变化的数据集,模型和系统。长期目标是通过连接研究人员和从业人员来加速创新,以便在https://cKnowledge.io上以通用,便携和可复制的格式共享和重用他们所有的知识,最佳实践,工件,工作流程和实验结果。
英文摘要:
This article provides the motivation and overview of the Collective Knowledge framework (CK or cKnowledge). The CK concept is to decompose research projects into reusable components that encapsulate research artifacts and provide unified application programming interfaces (APIs), command-line interfaces (CLIs), meta descriptions and common automation actions for related artifacts. The CK framework is used to organize and manage research projects as a database of such components. Inspired by the USB "plug and play" approach for hardware, CK also helps to assemble portable workflows that can automatically plug in compatible components from different users and vendors (models, datasets, frameworks, compilers, tools). Such workflows can build and run algorithms on different platforms and environments in a unified way using the universal CK program pipeline with software detection plugins and the automatic installation of missing packages. This article presents a number of industrial projects in which the modular CK approach was successfully validated in order to automate benchmarking, auto-tuning and co-design of efficient software and hardware for machine learning (ML) and artificial intelligence (AI) in terms of speed, accuracy, energy, size and various costs. The CK framework also helped to automate the artifact evaluation process at several computer science conferences as well as to make it easier to reproduce, compare and reuse research techniques from published papers, deploy them in production, and automatically adapt them to continuously changing datasets, models and systems. The long-term goal is to accelerate innovation by connecting researchers and practitioners to share and reuse all their knowledge, best practices, artifacts, workflows and experimental results in a common, portable and reproducible format at https://cKnowledge.io .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。