重新思考基于查询的Transformer用于持续图像分割
Rethinking Query-based Transformer for Continual Image Segmentation
AI总结：
针对持续图像分割中解耦框架的可塑性丧失和数据顺序依赖问题，提出SimCIS基线，通过直接选择图像特征进行查询分配以保持客观性并促进可塑性，并引入跨阶段一致性和视觉查询重放机制对抗灾难性遗忘，在多种任务和设置上优于现有方法。
摘要：
类增量/持续图像分割（CIS）旨在分阶段训练图像分割器，其中每个阶段可用的类别集合不同。为了利用基于查询的Transformer内置的客观性（以减轻掩码提议的灾难性遗忘），当前方法通常将掩码生成与持续学习过程解耦。然而，本研究发现了解耦框架的两个关键问题：可塑性丧失和对输入数据顺序的严重依赖。为了解决这些问题，我们深入调查了内置客观性，发现高度聚合的图像特征为查询通过简单的特征对齐生成掩码提供了捷径。基于此，我们提出了SimCIS，一个简单而强大的CIS基线。其核心思想是直接选择图像特征进行查询分配，确保“完美对齐”以保持客观性，同时允许查询选择新类别以促进可塑性。为了进一步对抗类别的灾难性遗忘，我们在选择中引入了跨阶段一致性，以及一种创新的基于“视觉查询”的重放机制。实验表明，SimCIS在各种分割任务、设置、数据划分和输入数据顺序上均优于最先进的方法。所有模型和代码将公开在https://github.com/SooLab/SimCIS。
英文摘要：
Class-incremental/Continual image segmentation (CIS) aims to train an image segmenter in stages, where the set of available categories differs at each stage. To leverage the built-in objectness of query-based transformers, which mitigates catastrophic forgetting of mask proposals, current methods often decouple mask generation from the continual learning process. This study, however, identifies two key issues with decoupled frameworks: loss of plasticity and heavy reliance on input data order. To address these, we conduct an in-depth investigation of the built-in objectness and find that highly aggregated image features provide a shortcut for queries to generate masks through simple feature alignment. Based on this, we propose SimCIS, a simple yet powerful baseline for CIS. Its core idea is to directly select image features for query assignment, ensuring "perfect alignment" to preserve objectness, while simultaneously allowing queries to select new classes to promote plasticity. To further combat catastrophic forgetting of categories, we introduce cross-stage consistency in selection and an innovative "visual query"-based replay mechanism. Experiments demonstrate that SimCIS consistently outperforms state-of-the-art methods across various segmentation tasks, settings, splits, and input data orders. All models and codes will be made publicly available at https://github.com/SooLab/SimCIS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
