在智能手机上作为实时的应用部署深度学习模型的指南和基准
Guidelines and Benchmarks for Deployment of Deep Learning Models on Smartphones as Real-Time Apps
摘要：
深度学习解决方案越来越多地用于移动应用程序。虽然有许多开源软件工具用于开发深度学习解决方案，但没有一个地方以统一的方式使用这些工具来在智能手机上实时的【形】部署这些解决方案。从各种可用的深度学习工具中，本文使用了最合适的工具，以便在智能手机上实时的【形】部署深度学习推理网络。为Android和iOS智能手机设计了统一的实现流程。还展示了使用多线程来实现或提高实时的【形】吞吐量的优势。一个基准框架，包括准确性，CPU/GPU的消耗和实时的【形】吞吐量被认为是验证的目的。开发的部署方法允许基于公开可用的深度学习和智能手机软件工具轻松将深度学习模型转化为实时的【形】智能手机应用程序。该方法被应用到六个流行的或有代表性的卷积神经网络模型，并报告了基于基准测试指标的验证结果。
英文摘要：
Deep learning solutions are being increasingly used in mobile applications. Although there are many open-source software tools for the development of deep learning solutions, there are no guidelines in one place in a unified manner for using these tools towards real-time deployment of these solutions on smartphones. From the variety of available deep learning tools, the most suited ones are used in this paper to enable real-time deployment of deep learning inference networks on smartphones. A uniform flow of implementation is devised for both Android and iOS smartphones. The advantage of using multi-threading to achieve or improve real-time throughputs is also showcased. A benchmarking framework consisting of accuracy, CPU/GPU consumption and real-time throughput is considered for validation purposes. The developed deployment approach allows deep learning models to be turned into real-time smartphone apps with ease based on publicly available deep learning and smartphone software tools. This approach is applied to six popular or representative convolutional neural network models and the validation results based on the benchmarking metrics are reported.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
