MCM：用于从掩码图像高效概念学习的多层概念图
MCM: Multi-layer Concept Map for Efficient Concept Learning from Masked Images
AI总结：
针对概念学习中掩码图像应用不足的问题，提出首个基于掩码图像的多层概念图（MCM）方法，通过非对称架构关联编解码层、用重建梯度更新多粒度概念令牌，降低计算成本且提升性能，还支持目标图像生成与比例可控的融合重建。
摘要：
自然语言处理中常用的掩码策略在概念学习等视觉任务中仍未得到充分探索，传统方法通常依赖完整图像。然而，使用掩码图像可丰富感知输入，有望为基于大规模Transformer模型的概念学习带来显著优势。为此，我们提出多层概念图（MCM），这是首个基于掩码图像设计的高效概念学习方法。具体而言，我们通过建立不同编码器和解码器层之间的关联，引入非对称概念学习架构，利用重建任务的反向梯度更新概念令牌。不同粒度级别的已学习概念令牌有助于填充掩码图像块以重建图像，或引导重建结果朝着反映特定概念的方向发展。此外，我们在多种指标上呈现了定量和定性结果，表明MCM通过仅使用不到75%的图像块进行训练，显著降低了计算成本，同时提升了概念预测性能。另外，在 latent 空间中编辑特定概念令牌可从掩码图像生成目标图像，使可见上下文块与提供的概念保持一致。通过进一步调整测试时的掩码比例，我们可生成一系列将可见块与提供的概念按所选比例融合的重建结果。
英文摘要：
Masking strategies commonly employed in natural language processing are still underexplored in vision tasks such as concept learning, where conventional methods typically rely on full images. However, using masked images diversifies perceptual inputs, potentially offering significant advantages in concept learning with large-scale Transformer models. To this end, we propose Multi-layer Concept Map (MCM), the first work to devise an efficient concept learning method based on masked images. In particular, we introduce an asymmetric concept learning architecture by establishing correlations between different encoder and decoder layers, updating concept tokens using backward gradients from reconstruction tasks. The learned concept tokens at various levels of granularity help either reconstruct the masked image patches by filling in gaps or guide the reconstruction results in a direction that reflects specific concepts. Moreover, we present both quantitative and qualitative results across a wide range of metrics, demonstrating that MCM significantly reduces computational costs by training on fewer than 75% of the total image patches while enhancing concept prediction performance. Additionally, editing specific concept tokens in the latent space enables targeted image generation from masked images, aligning both the visible contextual patches and the provided concepts. By further adjusting the testing time mask ratio, we could produce a range of reconstructions that blend the visible patches with the provided concepts, proportional to the chosen ratios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
