arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.05096cs.DCcs.LG
收藏

MLModelCI:高效MLaaS的自动化云平台

MLModelCI: An Automatic Cloud Platform for Efficient MLaaS

Nanyang Technological University(南洋理工大学);NVIDIA AI Tech Center(英伟达AI技术中心);Nokia Bell Labs(诺基亚贝尔实验室)

Huaizheng Zhang, Yuanming Li, Yizheng Huang, Yonggang Wen, Jianxiong Yin, Kyle Guan

摘要:

MLModelCI为多媒体研究人员和开发人员提供了一个高效的机器学习(ML)服务的一站式平台。该系统利用DevOps技术来优化、测试和管理模型。它还将这些优化和验证的模型作为云服务(MLaaS)进行容器化和部署。从本质上讲,MLModelCI是帮助用户发布模型的管家。模型首先被自动转换为用于生产目的的优化格式,然后在不同的设置(例如,批量大小和硬件)。分析信息可以用作平衡MLaaS性能和成本之间的权衡的指导原则。最后,系统将模型对接以便于部署到云环境。MLModelCI的一个关键特性是控制器的实现,它允许弹性评估,只利用空闲的工人,同时保持在线服务质量。我们的系统弥合了当前ML培训和服务系统之间的差距,从而将开发人员从通常与服务部署相关的手动和繁琐的工作中解放出来。我们在Apache 2.0许可下将该平台作为GitHub上的开源项目发布,旨在促进和简化更多大型ML应用程序和研究项目。

英文摘要:

MLModelCI provides multimedia researchers and developers with a one-stop platform for efficient machine learning (ML) services. The system leverages DevOps techniques to optimize, test, and manage models. It also containerizes and deploys these optimized and validated models as cloud services (MLaaS). In its essence, MLModelCI serves as a housekeeper to help users publish models. The models are first automatically converted to optimized formats for production purpose and then profiled under different settings (e.g., batch size and hardware). The profiling information can be used as guidelines for balancing the trade-off between performance and cost of MLaaS. Finally, the system dockerizes the models for ease of deployment to cloud environments. A key feature of MLModelCI is the implementation of a controller, which allows elastic evaluation which only utilizes idle workers while maintaining online service quality. Our system bridges the gap between current ML training and serving systems and thus free developers from manual and tedious work often associated with service deployment. We release the platform as an open-source project on GitHub under Apache 2.0 license, with the aim that it will facilitate and streamline more large-scale ML applications and research projects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑