arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.15423cs.CVcs.AIeess.IV
收藏

MR2-ByteTrack:基于CNN和Transformer的视频目标检测用于AI增强的嵌入式视觉传感器节点

MR2-ByteTrack: CNN and Transformer-based Video Object Detection for AI-augmented Embedded Vision Sensor Nodes

Electrical, Electronic and Information Engineering (DEI), University of Bologna, Italy.(博洛尼亚大学电气、电子与信息工程学院,意大利);Department of Electrical Engineering (ESAT), KU Leuven, Belgium.(卢旺达大学电气工程系,比利时);Dalle Molle Institute for Artificial Intelligence (IDSIA), USI--SUPSI, Switzerland.(人工智能研究所(IDSIA),瑞士USI--SUPSI)

Luca Bompani, Manuele Rusci, Luca Benini, Daniele Palossi, Francesco Conti

AI总结:

本文提出MR2-ByteTrack,一种针对嵌入式视觉节点的视频目标检测方法,通过交替使用全分辨率和低分辨率推理,结合ByteTrack和Rescore算法提升效率,实现在嵌入式设备上的高精度实时检测。

摘要:

现代智能视觉传感器需要设备端智能来处理视频流,因为云计算在带宽、延迟和隐私限制下往往不可行。然而,这些传感系统通常依赖超低功耗微控制器(MCUs),其内存和计算能力有限,使得需要特征存储或多帧缓冲的传统视频目标检测方法不可行。为了解决这一挑战,我们引入了多分辨率重评分ByteTrack(MR2-ByteTrack),一种专为基于MCU的嵌入式视觉节点设计的视频目标检测(VOD)方法。MR2-ByteTrack通过交替使用全分辨率和低分辨率推理来降低计算成本,同时通过ByteTrack在帧间链接检测,并通过Rescore算法通过概率联合规则聚合跨帧的检测置信度分数以纠正误分类。我们将其应用于基于CNN的检测器和基于Transformer的模型,证明了其在具有根本不同空间处理的架构中的通用性。在ImageNetVID上的实验表明,MR2-ByteTrack保持了准确性,实现了CNN模型的mAP最高达49.0,Transformer模型的mAP为48.7,同时将CNN的乘加操作减少了高达53%,Transformer的减少了32%。当部署在GAP9上,一个超低功耗RISC-V多核MCU上时,我们的方法相比仅处理全分辨率图像,实现了高达55%的能耗节省,实现了在MCU类嵌入式视觉节点上的首个实时Transformer-based VOD。代码可在https://github.com/Bomps4/Multi_Resolution_Rescored_ByteTrack/tree/IEEE_Access获取。

英文摘要:

Modern smart vision sensors need on-device intelligence to process video streams, as cloud computing is often impractical due to bandwidth, latency, and privacy constraints. However, these sensory systems typically rely on ultra-low-power microcontrollers (MCUs) with limited memory and compute, making conventional video object detection methods, which require feature storage or multi-frame buffering, unfeasible. To address this challenge, we introduce Multi-Resolution Rescored ByteTrack (MR2-ByteTrack), a Video Object Detection (VOD) method tailored for MCU-based embedded vision nodes. MR2-ByteTrack reduces computational cost by alternating between full- and low-resolution inference, while linking detections across frames via ByteTrack and correcting misclassifications through the Rescore algorithm, which applies probability union rules to aggregate detection confidence scores across frames. We apply our approach to both a CNN-based detector and a Transformer-based model, demonstrating its generality across architectures with fundamentally different spatial processing. Experiments on ImageNetVID demonstrate that MR2-ByteTrack maintains accuracy, achieving mAP scores of up to 49.0 for the CNN-based models and 48.7 for the Transformer, while reducing multiply-accumulate operations by as much as 53\% for the CNNs and 32\% for the Transformer. When deployed on GAP9, an ultra-low-power RISC-V multicore MCU, our method yields up to 55\% energy savings compared to processing only full-resolution images, enabling the first real-time Transformer-based VOD on an MCU-class embedded vision node. Code available at https://github.com/Bomps4/Multi_Resolution_Rescored_ByteTrack/tree/IEEE_Access

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑