arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.06402cs.LGcs.CVcs.DCstat.ML
收藏

SPINN:神经网络在设备和云上的协同渐进推理

SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud

Samsung AI Center, Cambridge(三星人工智能中心(剑桥));University of Cambridge(剑桥大学)

Stefanos Laskaridis, Stylianos I. Venieris, Mario Almeida, Ilias Leontiadis, Nicholas D. Lane

摘要:

尽管卷积神经网络(CNN)在移动应用中的使用激增,但由于现代CNN的计算需求过大以及部署设备的多样性日益增加,在移动设备上均匀地维持高性能推理一直是难以捉摸的。一个流行的替代方案包括将CNN处理卸载到强大的基于云的服务器。然而,通过依赖云来产生输出,新兴的关键任务和高移动性应用程序,如无人机避障或交互式应用程序,可能会受到云的动态连接条件和不确定可用性的影响。在本文中,我们提出了SPINN,一个分布式推理系统,采用协同设备云计算与渐进式推理方法,在不同的设置提供快速和强大的CNN推理。所提出的系统引入了一种新的调度器,该调度器在运行时间协同优化早期退出策略和CNN分裂,以适应动态条件并满足用户定义的服务级别要求。定量评估表明,SPINN在不同网络条件下实现的吞吐量方面优于其最先进的协同推理同行高达2倍,将服务器成本降低高达6.8倍,并在延迟限制下将准确性提高20.7%,同时在不确定的连接条件下提供稳健的操作,并与以云为中心的执行相比显著节省能源。

英文摘要:

Despite the soaring use of convolutional neural networks (CNNs) in mobile applications, uniformly sustaining high-performance inference on mobile has been elusive due to the excessive computational demands of modern CNNs and the increasing diversity of deployed devices. A popular alternative comprises offloading CNN processing to powerful cloud-based servers. Nevertheless, by relying on the cloud to produce outputs, emerging mission-critical and high-mobility applications, such as drone obstacle avoidance or interactive applications, can suffer from the dynamic connectivity conditions and the uncertain availability of the cloud. In this paper, we propose SPINN, a distributed inference system that employs synergistic device-cloud computation together with a progressive inference method to deliver fast and robust CNN inference across diverse settings. The proposed system introduces a novel scheduler that co-optimises the early-exit policy and the CNN splitting at run time, in order to adapt to dynamic conditions and meet user-defined service-level requirements. Quantitative evaluation illustrates that SPINN outperforms its state-of-the-art collaborative inference counterparts by up to 2x in achieved throughput under varying network conditions, reduces the server cost by up to 6.8x and improves accuracy by 20.7% under latency constraints, while providing robust operation under uncertain connectivity conditions and significant energy savings compared to cloud-centric execution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑