TFormer:一种适用于IoT器件的传输友好型ViT模型
TFormer: A Transmission-Friendly ViT Model for IoT Devices
摘要:
在无处不在的物联网(IoT)设备上部署高性能视觉Transformer(ViT)模型以提供高质量的视觉服务,将彻底改变我们生活、工作和与世界互动的方式。由于物联网设备资源有限与资源密集型ViT模型之间的矛盾,使用云服务器辅助ViT模型训练成为主流。然而,由于现有ViT模型的参数和浮点运算(FLOP)数量较多,云服务器传输的模型参数较大,难以在资源受限的物联网设备上运行。为此,本文提出了一种传输友好的ViT模型,TFormer,用于在云服务器的帮助下部署在资源受限的物联网设备上。TFormer的高性能和少量的模型参数和FLOP归因于所提出的混合层和所提出的部分连接前馈网络(PCS-FFN)。混合层由不可学习的模块和逐点卷积组成,它可以用很少的参数和FLOP来获得多类型和多尺度特征,以提高TFormer的性能。PCS-FFN采用组卷积来减少参数的数量。本文的核心思想是提出具有较少模型参数和FLOP的TFormer,以促进在资源受限的物联网设备上运行的应用程序从ViT模型的高性能中受益。在ImageNet-1 K、MS COCO和ADE 20 K数据集上进行图像分类、目标检测和语义分割任务的实验结果表明,该模型优于其他最先进的模型。具体来说,TFormer-S在ImageNet-1 K上的准确率比ResNet 18高5%,参数和FLOP少了1.4 $\times$。
英文摘要:
Deploying high-performance vision transformer (ViT) models on ubiquitous Internet of Things (IoT) devices to provide high-quality vision services will revolutionize the way we live, work, and interact with the world. Due to the contradiction between the limited resources of IoT devices and resource-intensive ViT models, the use of cloud servers to assist ViT model training has become mainstream. However, due to the larger number of parameters and floating-point operations (FLOPs) of the existing ViT models, the model parameters transmitted by cloud servers are large and difficult to run on resource-constrained IoT devices. To this end, this paper proposes a transmission-friendly ViT model, TFormer, for deployment on resource-constrained IoT devices with the assistance of a cloud server. The high performance and small number of model parameters and FLOPs of TFormer are attributed to the proposed hybrid layer and the proposed partially connected feed-forward network (PCS-FFN). The hybrid layer consists of nonlearnable modules and a pointwise convolution, which can obtain multitype and multiscale features with only a few parameters and FLOPs to improve the TFormer performance. The PCS-FFN adopts group convolution to reduce the number of parameters. The key idea of this paper is to propose TFormer with few model parameters and FLOPs to facilitate applications running on resource-constrained IoT devices to benefit from the high performance of the ViT models. Experimental results on the ImageNet-1K, MS COCO, and ADE20K datasets for image classification, object detection, and semantic segmentation tasks demonstrate that the proposed model outperforms other state-of-the-art models. Specifically, TFormer-S achieves 5% higher accuracy on ImageNet-1K than ResNet18 with 1.4$\times$ fewer parameters and FLOPs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。