先知注意力：利用未来注意力预测注意力以用于图像描述
Prophet Attention: Predicting Attention with Future Attention for Image Captioning
AI总结：
本文提出先知注意力模块，利用未来信息计算理想注意力权重以正则化解码中的焦点偏移问题，可嵌入现有图像描述模型，并在Flickr30k Entities和MSCOCO上取得最优性能。
摘要：
近年来，基于注意力的模型已被广泛应用于许多序列到序列学习系统中。特别是在图像描述任务中，基于注意力的模型被期望能够将正确的图像区域与适当生成的词语对应起来。然而，在解码过程的每个时间步中，基于注意力的模型通常使用当前输入的隐藏状态来关注图像区域。在这种设置下，这些注意力模型存在“焦点偏移”问题，即它们基于先前的词语而不是即将生成的词语来计算注意力权重，从而损害了图像区域定位和描述生成的性能。在本文中，我们提出了先知注意力（Prophet Attention），其形式类似于自监督。在训练阶段，该模块利用未来信息来计算针对图像区域的“理想”注意力权重。这些计算得到的“理想”权重进一步用于正则化“偏移”的注意力。通过这种方式，图像区域能够与正确的词语相对应。所提出的先知注意力可以很容易地集成到现有的图像描述模型中，以提升它们在图像区域定位和描述生成方面的性能。在Flickr30k Entities和MSCOCO数据集上的实验表明，所提出的先知注意力在自动评估指标和人工评估中均一致优于基线方法。值得注意的是，我们在两个基准数据集上取得了新的最优结果，并在在线MSCOCO基准的默认排名分数（即CIDEr-c40）上获得了排行榜第一名。
英文摘要：
Recently, attention based models have been used extensively in many sequence-to-sequence learning systems. Especially for image captioning, the attention based models are expected to ground correct image regions with proper generated words. However, for each time step in the decoding process, the attention based models usually use the hidden state of the current input to attend to the image regions. Under this setting, these attention models have a "deviated focus" problem that they calculate the attention weights based on previous words instead of the one to be generated, impairing the performance of both grounding and captioning. In this paper, we propose the Prophet Attention, similar to the form of self-supervision. In the training stage, this module utilizes the future information to calculate the "ideal" attention weights towards image regions. These calculated "ideal" weights are further used to regularize the "deviated" attention. In this manner, image regions are grounded with the correct words. The proposed Prophet Attention can be easily incorporated into existing image captioning models to improve their performance of both grounding and captioning. The experiments on the Flickr30k Entities and the MSCOCO datasets show that the proposed Prophet Attention consistently outperforms baselines in both automatic metrics and human evaluations. It is worth noticing that we set new state-of-the-arts on the two benchmark datasets and achieve the 1st place on the leaderboard of the online MSCOCO benchmark in terms of the default ranking score, i.e., CIDEr-c40.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
