Deeplite Neutrino:一个用于约束深度学习模型优化的端到端框架
Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization
摘要:
设计基于深度学习的解决方案正在成为训练具有更多层的更深模型的竞赛。虽然大型深度模型可以提供具有竞争力的准确性,但它在开发和部署过程中带来了许多后勤挑战和不合理的资源需求。这是深度学习模型在各种生产环境中没有被过度使用的关键原因之一,尤其是在边缘设备中。迫切需要优化和压缩这些深度学习模型,以实现设备上的智能。在这项研究中,我们引入了一个黑盒框架Deeplite Neutrino,用于深度学习模型的生产就绪优化。该框架为最终用户提供了一种简单的机制,以提供约束,例如优化模型的精度或目标大小的可容忍下降,以指导整个优化过程。该框架很容易包含在现有的生产管道中,并作为Python包提供,支持PyTorch和Tensorflow库。该框架的优化性能在多个基准数据集和流行的深度学习模型上得到了展示。此外,该框架目前已用于生产,并总结了来自几个客户的结果和推荐。
英文摘要:
Designing deep learning-based solutions is becoming a race for training deeper models with a greater number of layers. While a large-size deeper model could provide competitive accuracy, it creates a lot of logistical challenges and unreasonable resource requirements during development and deployment. This has been one of the key reasons for deep learning models not being excessively used in various production environments, especially in edge devices. There is an immediate requirement for optimizing and compressing these deep learning models, to enable on-device intelligence. In this research, we introduce a black-box framework, Deeplite Neutrino for production-ready optimization of deep learning models. The framework provides an easy mechanism for the end-users to provide constraints such as a tolerable drop in accuracy or target size of the optimized models, to guide the whole optimization process. The framework is easy to include in an existing production pipeline and is available as a Python Package, supporting PyTorch and Tensorflow libraries. The optimization performance of the framework is shown across multiple benchmark datasets and popular deep learning models. Further, the framework is currently used in production and the results and testimonials from several clients are summarized.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。