OODIn：一种面向异构移动设备的优化设备上推理框架
OODIn: An Optimised On-Device Inference Framework for Heterogeneous Mobile Devices
摘要：
深度学习（DL）领域的根本性进展使各种推理任务具有前所未有的准确性。因此，跨移动平台部署DL模型对于支持下一代智能应用程序的开发和广泛可用性至关重要。然而，DL模型的广泛和优化部署目前受到移动设备的巨大系统异构性、不同DL模型的不同计算成本以及DL应用中性能需求的可变性的阻碍。本文提出了OODIn，一个框架，优化部署的DL应用程序在异构移动设备。OODIn包括新颖的DL专用软件架构以及用于对DL应用进行建模的分析框架，其：（1）通过高度参数化的多层设计来抵消设备资源和DL模型中的可变性;以及（2）通过为DL推理应用程序设计的多目标公式对模型级和系统级参数进行原则性优化，以便使部署适应用户指定的性能要求和设备能力。定量评估表明，所提出的框架始终优于异构设备的现状设计，并提供高达4.3倍和3.5倍的性能增益分别超过高度优化的平台和模型感知设计，同时有效地适应执行资源可用性的动态变化。
英文摘要：
Radical progress in the field of deep learning (DL) has led to unprecedented accuracy in diverse inference tasks. As such, deploying DL models across mobile platforms is vital to enable the development and broad availability of the next-generation intelligent apps. Nevertheless, the wide and optimised deployment of DL models is currently hindered by the vast system heterogeneity of mobile devices, the varying computational cost of different DL models and the variability of performance needs across DL applications. This paper proposes OODIn, a framework for the optimised deployment of DL apps across heterogeneous mobile devices. OODIn comprises a novel DL-specific software architecture together with an analytical framework for modelling DL applications that: (1) counteract the variability in device resources and DL models by means of a highly parametrised multi-layer design; and (2) perform a principled optimisation of both model- and system-level parameters through a multi-objective formulation, designed for DL inference apps, in order to adapt the deployment to the user-specified performance requirements and device capabilities. Quantitative evaluation shows that the proposed framework consistently outperforms status-quo designs across heterogeneous devices and delivers up to 4.3x and 3.5x performance gain over highly optimised platform- and model-aware designs respectively, while effectively adapting execution to dynamic changes in resource availability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
