arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.12188cs.CVcs.AIcs.CL
收藏

基于WordEmbedding回归的深度解码器结构用于基于编码器-解码器的图像描述模型

A Deep Decoder Structure Based on WordEmbedding Regression for An Encoder-Decoder Based Model for Image Captioning

Amirkabir University of Technology(阿米尔卡比尔理工大学)

Ahmad Asadi, Reza Safabakhsh

摘要:

图像的文本描述是近年来计算机视觉和自然语言处理研究者关注的一个问题。为了解决这个问题,已经提出了数十种基于深度学习的模型。现有的方法是基于神经编码器-解码器结构配备的注意力机制。这些方法致力于训练解码器,以最小化给定前一个单词的句子中下一个单词的对数似然,这导致输出空间的稀疏性。在这项工作中,我们提出了一种新的方法来训练解码器回归的词嵌入的下一个字相对于以前的,而不是最小化的对数似然。所提出的方法能够学习和提取长期信息,并可以生成更长的细粒度字幕,而无需引入任何外部存储单元。此外,通过所提出的技术训练的解码器可以在生成字幕时考虑所生成的单词的重要性。此外,提出了一种新的语义注意力机制,通过图像引导注意点,考虑到先前生成的单词的含义。我们评估所提出的方法与MS-COCO数据集。所提出的模型优于最先进的模型,特别是在生成较长的字幕。它实现了CIDEr得分等于125.0和BLEU-4得分等于50.5,而最先进的模型的最佳得分分别为117.1和48.0。

英文摘要:

Generating textual descriptions for images has been an attractive problem for the computer vision and natural language processing researchers in recent years. Dozens of models based on deep learning have been proposed to solve this problem. The existing approaches are based on neural encoder-decoder structures equipped with the attention mechanism. These methods strive to train decoders to minimize the log likelihood of the next word in a sentence given the previous ones, which results in the sparsity of the output space. In this work, we propose a new approach to train decoders to regress the word embedding of the next word with respect to the previous ones instead of minimizing the log likelihood. The proposed method is able to learn and extract long-term information and can generate longer fine-grained captions without introducing any external memory cell. Furthermore, decoders trained by the proposed technique can take the importance of the generated words into consideration while generating captions. In addition, a novel semantic attention mechanism is proposed that guides attention points through the image, taking the meaning of the previously generated word into account. We evaluate the proposed approach with the MS-COCO dataset. The proposed model outperformed the state of the art models especially in generating longer captions. It achieved a CIDEr score equal to 125.0 and a BLEU-4 score equal to 50.5, while the best scores of the state of the art models are 117.1 and 48.0, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑