视频字幕生成中的堆叠注意力与语义硬拉取
Video captioning with stacked attention and semantic hard pull
AI总结：
本文提出语义敏感视频字幕生成（SSVC）架构，通过堆叠注意力和空间硬拉取改进上下文生成，并引入语义敏感性（SS）评分指标，提升视频字幕生成的性能。
摘要：
视频字幕生成，即为视频序列生成描述的任务，在计算机科学中搭建了自然语言处理与计算机视觉领域之间的桥梁。为视频生成语义准确的描述是一项相当复杂的任务。考虑到问题的复杂性，近期研究工作中获得的结果值得称赞。然而，仍有大量进一步研究的空间。本文针对这一空间提出了一种新颖的解决方案。大多数视频字幕生成模型包含两个顺序/循环层——一个作为视频到上下文的编码器，另一个作为上下文到字幕的解码器。本文提出了一种新颖的架构，即语义敏感视频字幕生成（SSVC），该架构通过使用两种新颖方法——“堆叠注意力”和“空间硬拉取”——来修改上下文生成机制。由于没有专门用于评估视频字幕生成模型的指标，我们强调对我们模型的定量和定性分析。因此，我们使用BLEU评分指标进行定量分析，并提出了一个用于定性分析的人工评估指标，即语义敏感性（SS）评分指标。SS评分克服了常见自动评分指标的缺点。本文报告称，上述新颖方法的使用提高了最先进架构的性能。
英文摘要：
Video captioning, i.e. the task of generating captions from video sequences creates a bridge between the Natural Language Processing and Computer Vision domains of computer science. The task of generating a semantically accurate description of a video is quite complex. Considering the complexity, of the problem, the results obtained in recent research works are praiseworthy. However, there is plenty of scope for further investigation. This paper addresses this scope and proposes a novel solution. Most video captioning models comprise two sequential/recurrent layers - one as a video-to-context encoder and the other as a context-to-caption decoder. This paper proposes a novel architecture, namely Semantically Sensible Video Captioning (SSVC) which modifies the context generation mechanism by using two novel approaches - "stacked attention" and "spatial hard pull". As there are no exclusive metrics for evaluating video captioning models, we emphasize both quantitative and qualitative analysis of our model. Hence, we have used the BLEU scoring metric for quantitative analysis and have proposed a human evaluation metric for qualitative analysis, namely the Semantic Sensibility (SS) scoring metric. SS Score overcomes the shortcomings of common automated scoring metrics. This paper reports that the use of the aforementioned novelties improves the performance of state-of-the-art architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
