请教专家：面向边缘加速器的视觉变换器协作推理
Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators
AI总结：
本文提出一种协作推理框架，通过边缘设备和近边缘加速器协同工作，提升视觉变换器在边缘计算中的性能与效率。
摘要：
在边缘设备上部署视觉变换器具有挑战性，因为它们的计算复杂度很高，而完全卸载到云资源则会产生显著的延迟开销。我们提出了一种新的协作推理框架，该框架在边缘设备上协调一个轻量级的通用视觉变换器（ViT），并在近边缘加速器上部署多个中等大小的专家ViT。一种新的路由机制利用边缘模型的Top-k预测，动态选择对置信度低的样本最相关的专家。我们进一步设计了一种渐进式专家训练策略，以提高专家在数据集子集上的准确性。在使用真实边缘和近边缘测试平台对CIFAR-100数据集进行的广泛实验中，证明了我们框架的优越性。具体来说，所提出的训练策略在目标子集上将专家专业化准确性提高了4.12%，并在静态专家上提高了2.76%的总体准确性。此外，我们的方法与边缘执行相比延迟减少了高达45%，与仅近边缘卸载相比能耗减少了高达46%。
英文摘要：
Deploying Vision Transformers on edge devices is challenging due to their high computational complexity, while full offloading to cloud resources presents significant latency overheads. We propose a novel collaborative inference framework, which orchestrates a lightweight generalist ViT on an edge device and multiple medium-sized expert ViTs on a near-edge accelerator. A novel routing mechanism uses the edge model's Top-$\mathit{k}$ predictions to dynamically select the most relevant expert for samples with low confidence. We further design a progressive specialist training strategy to enhance expert accuracy on dataset subsets. Extensive experiments on the CIFAR-100 dataset using a real-world edge and near-edge testbed demonstrate the superiority of our framework. Specifically, the proposed training strategy improves expert specialization accuracy by 4.12% on target subsets and enhances overall accuracy by 2.76% over static experts. Moreover, our method reduces latency by up to 45% compared to edge execution, and energy consumption by up to 46% compared to just near-edge offload.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
