kNN-CLIP: 检索实现持续扩展大词汇表下的免训练分割
kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabularies
AI总结：
针对传统持续训练存在灾难性遗忘的问题，提出免训练策略kNN-CLIP，通过实例嵌入数据库增强模型，实现零遗忘、低成本的大词汇表持续分割，性能达当前最优。
摘要：
持续分割尚未解决利用训练数据改进开放词汇分割模型、以在大规模持续扩展的词汇表上实现准确分割的挑战。我们发现传统持续训练会导致严重的灾难性遗忘，性能无法超过零样本分割基线。我们提出一种新颖的免训练策略kNN-CLIP，该策略通过实例嵌入数据库增强模型，用于语义分割和全景分割，实现零遗忘。我们证明kNN-CLIP无需重新训练或高昂内存成本，即可适应不断增长的词汇表。kNN-CLIP让开放词汇分割方法仅需单次遍历数据，就能在任意领域扩展词汇表，同时仅存储紧凑的嵌入向量。该方法将计算和内存成本降至最低。kNN-CLIP在大词汇表语义分割和全景分割数据集上取得了当前最优性能。我们希望kNN-CLIP能成为实现更高效、更具适应性的持续分割的重要一步，为现实世界大词汇表持续分割方法的进展铺平道路。
英文摘要：
Continual segmentation has not yet tackled the challenge of improving open-vocabulary segmentation models with training data for accurate segmentation across large, continually expanding vocabularies. We discover that traditional continual training results in severe catastrophic forgetting, failing to outperform a zero-shot segmentation baseline. We introduce a novel training-free strategy, kNN-CLIP, which augments the model with a database of instance embeddings for semantic and panoptic segmentation that achieves zero forgetting. We demonstrate that kNN-CLIP can adapt to continually growing vocabularies without the need for retraining or large memory costs. kNN-CLIP enables open-vocabulary segmentation methods to expand their vocabularies on any domain with a single pass through the data, while only storing compact embeddings. This approach minimizes both compute and memory costs. kNN-CLIP achieves state-of-the-art performance across large-vocabulary semantic and panoptic segmentation datasets. We hope kNN-CLIP represents a significant step forward in enabling more efficient and adaptable continual segmentation, paving the way for advances in real-world large-vocabulary continual segmentation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
