MultiTASC++:一种用于边缘多设备级联推理的持续自适应调度器
MultiTASC++: A Continuously Adaptive Scheduler for Edge-Based Multi-Device Cascade Inference
AI总结:
针对多设备共享边缘服务器重型模型的级联推理场景,本文提出 MultiTASC++ 自适应多租户调度器,动态调整转发决策以兼顾吞吐量、精度、延迟和满意率。
摘要:
级联系统由处理所有样本的轻量级模型和精炼困难样本的更重、高精度模型组成,已成为一种被广泛采用的分布式推理方法,可在为移动和物联网设备保持低计算负担的同时实现高精度。随着智能家居等智能室内环境不断扩展,一种新场景出现了,即多设备级联。在这种设置中,多个不同设备同时使用托管在服务器上的共享重型模型,该服务器通常位于消费者环境内或邻近环境。本工作提出 MultiTASC++,这是一种持续自适应、感知多租户的调度器,可动态控制各设备的转发决策函数,以在保持高精度和低延迟的同时优化系统吞吐量。通过在不同设备环境中以及使用不同服务器端模型进行大量实验,我们证明该调度器能够持续维持目标满意率,并在不同设备层级和最高达100台设备的工作负载下提供最高的可用精度。这表明了其在应对动态、多样物联网环境中协作式 DNN 推理独特挑战时的可扩展性和效率。
英文摘要:
Cascade systems, consisting of a lightweight model processing all samples and a heavier, high-accuracy model refining challenging samples, have become a widely-adopted distributed inference approach to achieving high accuracy and maintaining a low computational burden for mobile and IoT devices. As intelligent indoor environments, like smart homes, continue to expand, a new scenario emerges, the multi-device cascade. In this setting, multiple diverse devices simultaneously utilize a shared heavy model hosted on a server, often situated within or close to the consumer environment. This work introduces MultiTASC++, a continuously adaptive multi-tenancy-aware scheduler that dynamically controls the forwarding decision functions of devices to optimize system throughput while maintaining high accuracy and low latency. Through extensive experimentation in diverse device environments and with varying server-side models, we demonstrate the scheduler's efficacy in consistently maintaining a targeted satisfaction rate while providing the highest available accuracy across different device tiers and workloads of up to 100 devices. This demonstrates its scalability and efficiency in addressing the unique challenges of collaborative DNN inference in dynamic and diverse IoT environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。