arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.04123eess.IV
收藏

面向云边多模态3D目标检测的特征压缩

Feature Compression for Cloud-Edge Multimodal 3D Object Detection

Chongzhen Tian, Zhengxin Li, Hui Yuan, Raouf Hamzaoui, Liquan Shen, Sam Kwong

AI总结:

针对云边协同多模态3D目标检测的特征传输瓶颈,提出T-FFC与A-FFC两种压缩模式,在KITTI上分别实现6061倍和901倍压缩并保持检测精度。

摘要:

能够高效处理大量视觉感知任务的机器视觉系统,在工业生产和日常生活中日益普及。由于单个传感器难以同时获取准确的深度与纹理信息,相机和LiDAR采集的多模态数据通常被用于提升性能。此外,云边协同已成为一种新型计算方式,可在机器视觉系统中改善用户体验并保障数据安全。本文提出一种开创性方案,以解决多模态3D目标检测中的特征压缩问题。针对边缘设备上基于稀疏张量的目标检测网络,我们引入两种模式以适应不同应用需求:Transmission-Friendly Feature Compression(T-FFC,传输友好型特征压缩)和Accuracy-Friendly Feature Compression(A-FFC,精度友好型特征压缩)。在T-FFC模式下,边缘设备仅传输网络主干最后一层的输出。云端设备通过一个通道扩展模块和两个空间上采样模块处理接收到的特征,以生成多尺度特征。在A-FFC模式下,我们在T-FFC模式基础上扩展,额外传输两类特征。这些新增特征使云端设备能够生成更准确的多尺度特征。在KITTI数据集上使用VirConv-L检测网络进行的实验结果表明,T-FFC能够将特征压缩6061倍,且检测性能下降不到3%。另一方面,A-FFC将特征压缩约901倍,检测性能几乎没有下降。我们还设计了可选的残差提取和3D目标重建模块,以促进对检测到的目标进行重建。重建目标有效反映了原始目标的细节。

英文摘要:

Machine vision systems, which can efficiently manage extensive visual perception tasks, are becoming increasingly popular in industrial production and daily life. Due to the challenge of simultaneously obtaining accurate depth and texture information with a single sensor, multimodal data captured by cameras and LiDAR is commonly used to enhance performance. Additionally, cloud-edge cooperation has emerged as a novel computing approach to improve user experience and ensure data security in machine vision systems. This paper proposes a pioneering solution to address the feature compression problem in multimodal 3D object detection. Given a sparse tensor-based object detection network at the edge device, we introduce two modes to accommodate different application requirements: Transmission-Friendly Feature Compression (T-FFC) and Accuracy-Friendly Feature Compression (A-FFC). In T-FFC mode, only the output of the last layer of the network's backbone is transmitted from the edge device. The received feature is processed at the cloud device through a channel expansion module and two spatial upsampling modules to generate multi-scale features. In A-FFC mode, we expand upon the T-FFC mode by transmitting two additional types of features. These added features enable the cloud device to generate more accurate multi-scale features. Experimental results on the KITTI dataset using the VirConv-L detection network showed that T-FFC was able to compress the features by a factor of 6061 with less than a 3% reduction in detection performance. On the other hand, A-FFC compressed the features by a factor of about 901 with almost no degradation in detection performance. We also designed optional residual extraction and 3D object reconstruction modules to facilitate the reconstruction of detected objects. The reconstructed objects effectively reflected details of the original objects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑