PatchCT：面向多标签图像分类，利用条件传输对齐补丁集与标签集
PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification
AI总结：
本文提出 PatchCT，将多标签图像分类建模为条件传输问题，通过双向 CT 代价对齐图像补丁嵌入与标签嵌入，并在三个公开基准上持续超越以往方法。
摘要：
多标签图像分类是一项旨在从给定图像中识别出多个标签的预测任务。本文关注视觉补丁域与语言标签域之间潜在空间的语义一致性，并引入条件传输（CT）理论来弥合这一公认的鸿沟。尽管近期基于跨模态注意力的研究已尝试对齐这两种表示并取得了令人瞩目的性能，但它们需要精心设计的对齐模块以及注意力计算中额外的复杂操作。我们发现，通过将多标签分类表述为一个 CT 问题，可以利用最小化双向 CT 代价来高效挖掘图像与标签之间的交互。具体而言，在将图像和文本标签输入特定模态的编码器之后，我们将每张图像视为补丁嵌入的混合，并将标签视为标签嵌入的混合，二者分别捕获局部区域特征和类别原型。随后采用 CT，通过定义前向和后向导航器来学习并对齐这两个语义集合。重要的是，CT 距离中定义的导航器对补丁与标签之间的相似性进行建模，从而提供了一种可解释的工具来可视化所学习的原型。在三个公开图像基准上进行的大量实验表明，所提出的模型始终优于以往方法。
英文摘要：
Multi-label image classification is a prediction task that aims to identify more than one label from a given image. This paper considers the semantic consistency of the latent space between the visual patch and linguistic label domains and introduces the conditional transport (CT) theory to bridge the acknowledged gap. While recent cross-modal attention-based studies have attempted to align such two representations and achieved impressive performance, they required carefully-designed alignment modules and extra complex operations in the attention computation. We find that by formulating the multi-label classification as a CT problem, we can exploit the interactions between the image and label efficiently by minimizing the bidirectional CT cost. Specifically, after feeding the images and textual labels into the modality-specific encoders, we view each image as a mixture of patch embeddings and a mixture of label embeddings, which capture the local region features and the class prototypes, respectively. CT is then employed to learn and align those two semantic sets by defining the forward and backward navigators. Importantly, the defined navigators in CT distance model the similarities between patches and labels, which provides an interpretable tool to visualize the learned prototypes. Extensive experiments on three public image benchmarks show that the proposed model consistently outperforms the previous methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
