WARM-CAT: 基于温启动的测试时综合知识积累用于组合零样本学习
WARM-CAT: Warm-Started Test-Time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning
AI总结：
WARM-CAT通过温启动测试时综合知识积累，提升组合零样本学习的性能。
摘要：
组合零样本学习（CZSL）旨在基于已学习的知识来识别新的属性-对象组合。现有方法由于测试时标签空间的分布偏移而出现性能下降，这源于包含未见组合的重新组合。为克服这一挑战，我们提出了一种新的方法，从无监督数据中在文本和视觉模态中积累综合知识，以在测试时更新多模态原型。在此基础上，我们进一步设计了一个自适应更新权重，以控制原型调整的程度，使模型能够灵活适应测试时的分布偏移。此外，引入了一个动态优先队列，存储高置信度图像，以从历史图像中获取视觉原型用于推理。由于模型在测试时倾向于选择队列中已存储的组合，我们通过初始化队列以存储训练图像的视觉原型来预热队列，并利用已见和未见文本原型之间的映射生成未见视觉原型。考虑到多模态知识的语义一致性，我们通过多模态协作表征学习对文本和视觉原型进行对齐。为了提供更可靠的CZSL评估，我们引入了一个新的基准数据集C-Fashion，并优化了广泛使用的但嘈杂的MIT-States数据集。广泛的实验表明，我们的方法在四个基准数据集上在封闭世界和开放世界设置下均实现了最先进的性能。源代码和数据集可在https://github.com/xud-yan/WARM-CAT上获得。
英文摘要：
Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones. Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data to update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, a dynamic priority queue is introduced that stores high-confidence images to acquire visual prototypes from historical images for inference. Since the model tends to favor compositions already stored in the queue during testing, we warm-start the queue by initializing it with training images for visual prototypes of seen compositions and generating unseen visual prototypes using the mapping learned between seen and unseen textual prototypes. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. To provide a more reliable evaluation for CZSL, we introduce a new benchmark dataset, C-Fashion, and refine the widely used but noisy MIT-States dataset. Extensive experiments indicate that our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings. The source code and datasets are available at https://github.com/xud-yan/WARM-CAT .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
