Arena:一种用于边缘辅助视频分析的感兴趣块ViT推理加速系统
Arena: A Patch-of-Interest ViT Inference Acceleration System for Edge-Assisted Video Analytics
AI总结:
本文提出Arena,一个基于ViT的边缘辅助视频推理加速系统,通过token剪枝仅处理感兴趣块并设计自适应关键帧推理切换算法,在保持高准确率的同时显著提升了推理速度并节省带宽。
摘要:
边缘计算的出现使得实时智能视频分析成为可能。先前基于传统模型架构(如CNN、RNN等)的工作采用各种策略过滤非感兴趣区域内容,以最小化带宽和计算消耗,但在恶劣环境中表现不佳。最近,基于transformer的视觉基础模型因其惊人的泛化能力在恶劣环境中展现出优异性能。然而,它们需要大量计算能力,这限制了其在实时智能视频分析中的应用。在本文中,我们发现像Vision Transformer(ViT)这样的视觉基础模型也具有专门用于视频分析的加速机制。为此,我们引入了Arena,一个基于ViT的端到端边缘辅助视频推理加速系统。我们利用ViT可通过token剪枝加速的能力,仅将感兴趣块卸载并输入给下游模型。此外,我们设计了一种针对不同视频量身定制的自适应关键帧推理切换算法,能够适应当前视频内容以联合优化准确率和带宽。通过大量实验,我们的研究结果表明,Arena能将推理速度分别提升最高1.58倍和平均1.82倍,同时仅消耗47%和31%的带宽,且均保持高推理准确率。
英文摘要:
The advent of edge computing has made real-time intelligent video analytics feasible. Previous works, based on traditional model architecture (e.g., CNN, RNN, etc.), employ various strategies to filter out non-region-of-interest content to minimize bandwidth and computation consumption but show inferior performance in adverse environments. Recently, visual foundation models based on transformers have shown great performance in adverse environments due to their amazing generalization capability. However, they require a large amount of computation power, which limits their applications in real-time intelligent video analytics. In this paper, we find visual foundation models like Vision Transformer (ViT) also have a dedicated acceleration mechanism for video analytics. To this end, we introduce Arena, an end-to-end edge-assisted video inference acceleration system based on ViT. We leverage the capability of ViT that can be accelerated through token pruning by only offloading and feeding Patches-of-Interest to the downstream models. Additionally, we design an adaptive keyframe inference switching algorithm tailored to different videos, capable of adapting to the current video content to jointly optimize accuracy and bandwidth. Through extensive experiments, our findings reveal that Arena can boost inference speeds by up to 1.58\(\times\) and 1.82\(\times\) on average while consuming only 47\% and 31\% of the bandwidth, respectively, all with high inference accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。