MAMBA：基于记忆库的多层聚合用于视频目标检测
MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection
AI总结：
针对现有视频目标检测记忆结构计算成本高、时序信息捕获不足的问题，提出基于记忆库的多层聚合架构MAMBA，结合轻量键集构建、细粒度特征更新与通用增强操作，在ImageNetVID数据集上实现了速度与精度的双重提升。
摘要：
当前最先进的视频目标检测方法通常采用滑动窗口或记忆队列的记忆结构，通过注意力机制增强当前帧特征。但我们认为这类记忆结构效率不足且信息利用不充分，源于两个隐含操作：一是拼接记忆中所有特征用于增强，导致计算成本极高；二是逐帧更新记忆，限制了记忆对更多时序信息的捕获。本文提出一种基于记忆库的多层聚合架构MAMBA。具体而言，我们的记忆库采用两种全新操作以消除现有方法的弊端：一是轻量级键集构建，可显著降低计算成本；二是细粒度的特征级更新策略，使方法能够利用来自整个视频的知识。为更好地增强来自互补层级（即特征图和候选框）的特征，我们进一步提出通用增强操作（GEO），以统一方式聚合多层特征。我们在极具挑战性的ImageNetVID数据集上开展了广泛评估。与现有最先进方法相比，本文方法在速度和精度上均实现了更优性能。更值得注意的是，基于ResNet-101的MAMBA在12.6/9.1 FPS的速度下达到了83.7/84.6%的mAP。代码可在https://github.com/guanxiongsun/vfe.pytorch获取。
英文摘要：
State-of-the-art video object detection methods maintain a memory structure, either a sliding window or a memory queue, to enhance the current frame using attention mechanisms. However, we argue that these memory structures are not efficient or sufficient because of two implied operations: (1) concatenating all features in memory for enhancement, leading to a heavy computational cost; (2) frame-wise memory updating, preventing the memory from capturing more temporal information. In this paper, we propose a multi-level aggregation architecture via memory bank called MAMBA. Specifically, our memory bank employs two novel operations to eliminate the disadvantages of existing methods: (1) light-weight key-set construction which can significantly reduce the computational cost; (2) fine-grained feature-wise updating strategy which enables our method to utilize knowledge from the whole video. To better enhance features from complementary levels, i.e., feature maps and proposals, we further propose a generalized enhancement operation (GEO) to aggregate multi-level features in a unified manner. We conduct extensive evaluations on the challenging ImageNetVID dataset. Compared with existing state-of-the-art methods, our method achieves superior performance in terms of both speed and accuracy. More remarkably, MAMBA achieves mAP of 83.7/84.6% at 12.6/9.1 FPS with ResNet-101. Code is available at https://github.com/guanxiongsun/vfe.pytorch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
