实时学习:基于回放的连续物体感知用于室内无人机
Learning on the Fly: Replay-Based Continual Object Perception for Indoor Drones
AI总结:
本研究提出基于回放的连续学习方法,通过室内无人机视频数据集评估,证明在有限内存预算下FAR方法能有效提升物体感知性能。
摘要:
自主代理如室内无人机必须在实时中学习新的物体类别,同时限制灾难性遗忘,这推动了类增量学习(CIL)。然而,大多数无人 aerial 车(UAV)数据集专注于户外场景,且提供有限的时空一致的室内视频。我们介绍了一个包含14400帧的室内数据集,捕捉了无人机和地面车辆的 footage,通过半自动工作流程进行标注,首次通过标注一致性达到98.6%后,再进行最终手动验证。使用此数据集,我们评估了三种基于回放的CIL策略:经验回放(ER)、最大干扰检索(MIR)和遗忘感知回放(FAR),使用YOLOv11-nano作为资源高效的检测器用于部署受限的UAV平台。在紧绷的内存预算(5-10%回放)下,FAR优于其他方法,达到平均准确率(ACC,跨增量的mAP50-95)为82.96%,使用5%回放。梯度加权类激活映射(Grad-CAM)分析显示,在混合场景中注意力在不同类别之间转移,这与无人机的定位质量下降有关。实验进一步证明,基于回放的持续学习可以有效应用于边缘空中的系统。总体而言,这项工作贡献了一个具有保留时间一致性的小型UAV视频数据集,并在有限的回放预算下评估了基于回放的CIL。项目页面:https://spacetime-vision-robotics-laboratory.github.io/learning-on-the-fly-cl
英文摘要:
Autonomous agents such as indoor drones must learn new object classes in real-time while limiting catastrophic forgetting, motivating Class-Incremental Learning (CIL). However, most unmanned aerial vehicle (UAV) datasets focus on outdoor scenes and offer limited temporally coherent indoor videos. We introduce an indoor dataset of $14,400$ frames capturing inter-drone and ground vehicle footage, annotated via a semi-automatic workflow with a $98.6\%$ first-pass labeling agreement before final manual verification. Using this dataset, we benchmark 3 replay-based CIL strategies: Experience Replay (ER), Maximally Interfered Retrieval (MIR), and Forgetting-Aware Replay (FAR), using YOLOv11-nano as a resource-efficient detector for deployment-constrained UAV platforms. Under tight memory budgets ($5-10\%$ replay), FAR performs better than the rest, achieving an average accuracy (ACC, $mAP_{50-95}$ across increments) of $82.96\%$ with $5\%$ replay. Gradient-weighted class activation mapping (Grad-CAM) analysis shows attention shifts across classes in mixed scenes, which is associated with reduced localization quality for drones. The experiments further demonstrate that replay-based continual learning can be effectively applied to edge aerial systems. Overall, this work contributes an indoor UAV video dataset with preserved temporal coherence and an evaluation of replay-based CIL under limited replay budgets. Project page: https://spacetime-vision-robotics-laboratory.github.io/learning-on-the-fly-cl
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。