RevCD——用于广义零样本学习的反向条件扩散模型
RevCD -- Reversed Conditional Diffusion for Generalized Zero-Shot Learning
AI总结：
针对广义零样本学习中联合分布学习成本高的问题，提出反向条件扩散模型RevCD，通过视觉输入生成语义特征，含三项创新，经跨数据集验证性能，代码将开源。
摘要：
在广义零样本学习（GZSL）中，我们的目标是使用仅在已见类别上训练的模型识别已见和未见类别。在计算机视觉领域，这转化为一个分类问题，即通过利用视觉特征与文本语料库或人工标注等可用语义信息之间的关系，将已见类别的知识迁移到未见类别。然而，学习这种联合分布成本高昂，且需要与对应语义信息进行一对一训练。我们提出了一种基于反向条件扩散的模型（RevCD），该模型利用扩散模型的条件机制，通过从视觉输入合成生成语义特征来缓解这一问题。我们的RevCD模型由正弦时间调度的交叉哈达玛-加法嵌入和用于注意力引导嵌入的多头视觉Transformer组成。该方法有三项关键创新：第一，我们反转了基于视觉数据生成语义空间的过程，引入了一种新型损失函数以实现更高效的知识迁移；第二，我们将扩散模型应用于零样本学习——这是一种利用其捕捉数据复杂性优势的新方法；第三，我们通过全面的跨数据集评估验证了模型的性能。完整代码将在GitHub上发布。
英文摘要：
In Generalized Zero-Shot Learning (GZSL), we aim to recognize both seen and unseen categories using a model trained only on seen categories. In computer vision, this translates into a classification problem, where knowledge from seen categories is transferred to unseen categories by exploiting the relationships between visual features and available semantic information, such as text corpora or manual annotations. However, learning this joint distribution is costly and requires one-to-one training with corresponding semantic information. We present a reversed conditional Diffusion-based model (RevCD) that mitigates this issue by generating semantic features synthesized from visual inputs by leveraging Diffusion models' conditional mechanisms. Our RevCD model consists of a cross Hadamard-Addition embedding of a sinusoidal time schedule and a multi-headed visual transformer for attention-guided embeddings. The proposed approach introduces three key innovations. First, we reverse the process of generating semantic space based on visual data, introducing a novel loss function that facilitates more efficient knowledge transfer. Second, we apply Diffusion models to zero-shot learning - a novel approach that exploits their strengths in capturing data complexity. Third, we demonstrate our model's performance through a comprehensive cross-dataset evaluation. The complete code will be available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
