arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.14797cs.CVcs.LG
收藏

在连续检测中通过记忆网络防止灾难性遗忘

Preventing Catastrophic Forgetting through Memory Networks in Continuous Detection

University of British Columbia(不列颠哥伦比亚大学);The Vector Institute for AI(矢量人工智能研究所)

Gaurav Bhatt, James Ross, Leonid Sigal

AI总结:

针对连续微调下检测模型的灾难性遗忘及背景降级问题,提出带局部化查询的记忆型DETR架构,在MS-COCO等基准上较现有最优方法提升5-7%。

摘要:

现代预训练架构在针对新任务进行连续微调时,难以保留先前习得的信息。尽管持续分类领域已取得显著进展,但面向检测、分割等复杂视觉任务设计的系统仍难以达到令人满意的性能。本研究提出一种基于记忆的检测Transformer架构,可使预训练的DETR(检测Transformer)风格检测器适配新任务,同时保留过往任务的知识。我们设计了一种新型局部化查询函数,用于从记忆单元中高效检索信息,以最大程度降低遗忘程度。此外,我们发现了持续检测中的一项核心挑战,称之为“背景降级”:当早期任务的物体类别在后续任务中再次出现(可能未被标注)时,会被隐式当作背景处理,这是持续检测或分割任务中不可避免的问题。所提出的持续优化技术可有效应对这一挑战。最后,我们在多个持续检测基准上评估了所提系统的性能,结果表明该方法优于现有最优方法,在MS-COCO和PASCAL-VOC数据集的持续检测任务上实现了5-7%的性能提升。

英文摘要:

Modern pre-trained architectures struggle to retain previous information while undergoing continuous fine-tuning on new tasks. Despite notable progress in continual classification, systems designed for complex vision tasks such as detection or segmentation still struggle to attain satisfactory performance. In this work, we introduce a memory-based detection transformer architecture to adapt a pre-trained DETR-style detector to new tasks while preserving knowledge from previous tasks. We propose a novel localized query function for efficient information retrieval from memory units, aiming to minimize forgetting. Furthermore, we identify a fundamental challenge in continual detection referred to as background relegation. This arises when object categories from earlier tasks reappear in future tasks, potentially without labels, leading them to be implicitly treated as background. This is an inevitable issue in continual detection or segmentation. The introduced continual optimization technique effectively tackles this challenge. Finally, we assess the performance of our proposed system on continual detection benchmarks and demonstrate that our approach surpasses the performance of existing state-of-the-art resulting in 5-7% improvements on MS-COCO and PASCAL-VOC on the task of continual detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑