arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.10541cs.LGcs.AI
收藏

多用户端边缘云网络中推理演示的在线学习

Online Learning for Orchestration of Inference in Multi-User End-Edge-Cloud Networks

University of California, Irvine(加州大学欧文分校);University of Turku(图尔库大学);Kookmin University(国民大学);San Diego State University(圣地亚哥州立大学)

Sina Shahhosseini, Dongjoo Seo, Anil Kanduri, Tianyi Hu, Sung-soo Lim, Bryan Donyanavard, Amir M. Rahmani, Nikil Dutt

摘要:

基于深度学习的智能服务在包括智能城市和医疗保健在内的网络物理应用中已经变得普遍。在终端用户附近部署基于深度学习的智能可以增强隐私保护、响应能力和可靠性。必须仔细管理资源受限的终端设备,以满足计算密集型深度学习服务的延迟和能量要求。用于深度学习的协作式端边缘云计算提供了一系列性能和效率,可通过计算卸载来满足应用程序需求。卸载计算的决定是随着两个系统参数(例如,网络条件)和工作负载特性(例如,输入)。另一方面,深度学习模型优化提供了延迟和模型准确性之间的另一个折衷来源。需要考虑这种计算-通信问题的端到端决策解决方案来协同地找到用于深度学习服务的最佳卸载策略和模型。为此,我们提出了一种基于学习的计算卸载解决方案,该解决方案考虑到深度学习模型选择技术来学习最佳卸载策略,以最小化响应时间,同时提供足够的准确性。我们在端-边缘-云系统中展示了我们的边缘设备解决方案的有效性,并使用多个AWS和ARM核心配置进行了实际设置实施评估。与最先进的解决方案相比,我们的解决方案在平均响应时间方面提供了35%的加速,准确性降低不到0.9%,证明了我们的在线学习框架在端边缘云系统中编排DL推理的前景。

英文摘要:

Deep-learning-based intelligent services have become prevalent in cyber-physical applications including smart cities and health-care. Deploying deep-learning-based intelligence near the end-user enhances privacy protection, responsiveness, and reliability. Resource-constrained end-devices must be carefully managed in order to meet the latency and energy requirements of computationally-intensive deep learning services. Collaborative end-edge-cloud computing for deep learning provides a range of performance and efficiency that can address application requirements through computation offloading. The decision to offload computation is a communication-computation co-optimization problem that varies with both system parameters (e.g., network condition) and workload characteristics (e.g., inputs). On the other hand, deep learning model optimization provides another source of tradeoff between latency and model accuracy. An end-to-end decision-making solution that considers such computation-communication problem is required to synergistically find the optimal offloading policy and model for deep learning services. To this end, we propose a reinforcement-learning-based computation offloading solution that learns optimal offloading policy considering deep learning model selection techniques to minimize response time while providing sufficient accuracy. We demonstrate the effectiveness of our solution for edge devices in an end-edge-cloud system and evaluate with a real-setup implementation using multiple AWS and ARM core configurations. Our solution provides 35% speedup in the average response time compared to the state-of-the-art with less than 0.9% accuracy reduction, demonstrating the promise of our online learning framework for orchestrating DL inference in end-edge-cloud systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑