用于图像描述的堆叠跨模态特征整合注意力网络
Stacked Cross-modal Feature Consolidation Attention Networks for Image Captioning
AI总结：
针对图像描述中视觉到文本直接转换难以生成细粒度描述的问题，提出SCFC注意力网络与SCFC-LSTM，通过多步复合融合空间信息和CAA，并在MSCOCO、Flickr30K上优于现有方法。
摘要：
近来，富含注意力机制的编码器-解码器框架因其令人瞩目的进展，在图像描述领域引起了广泛关注。许多视觉注意力模型直接利用有意义的区域来生成图像描述。然而，寻求从视觉空间到文本的直接转换并不足以生成细粒度的描述。本文采用一种特征复合方法，以完全端到端的方式将有关上下文环境的高级语义概念与视觉信息结合起来。因此，我们提出了一种用于图像描述的堆叠跨模态特征整合（SCFC）注意力网络，在该网络中，我们通过一种新颖的复合函数，以多步推理的方式同时整合跨模态特征。此外，我们在所提出的复合函数中联合采用空间信息和上下文感知属性（CAA）作为主要组成部分，其中CAA提供简洁的上下文敏感语义表示。为了更好地利用整合特征的潜力，我们进一步提出SCFC-LSTM作为描述生成器，它能够在描述生成过程中利用具有判别性的语义信息。实验结果表明，在MSCOCO和Flickr30K数据集上，就常用指标而言，我们提出的SCFC能够优于多种最先进的图像描述基准方法。
英文摘要：
Recently, the attention-enriched encoder-decoder framework has aroused great interest in image captioning due to its overwhelming progress. Many visual attention models directly leverage meaningful regions to generate image descriptions. However, seeking a direct transition from visual space to text is not enough to generate fine-grained captions. This paper exploits a feature-compounding approach to bring together high-level semantic concepts and visual information regarding the contextual environment fully end-to-end. Thus, we propose a stacked cross-modal feature consolidation (SCFC) attention network for image captioning in which we simultaneously consolidate cross-modal features through a novel compounding function in a multi-step reasoning fashion. Besides, we jointly employ spatial information and context-aware attributes (CAA) as the principal components in our proposed compounding function, where our CAA provides a concise context-sensitive semantic representation. To make better use of consolidated features potential, we further propose an SCFC-LSTM as the caption generator, which can leverage discriminative semantic information through the caption generation process. The experimental results indicate that our proposed SCFC can outperform various state-of-the-art image captioning benchmarks in terms of popular metrics on the MSCOCO and Flickr30K datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
