通过预训练将事件检测与字幕生成统一为序列生成任务
Unifying Event Detection and Captioning as Sequence Generation via Pre-Training
AI总结：
针对密集视频字幕生成中事件检测与字幕生成任务关联弱、事件检测忽略时序依赖的问题，提出将事件检测定义为序列生成任务的统一预训练微调框架，在ActivityNet数据集上性能超越SOTA，预训练后可进一步提升。
摘要：
密集视频字幕生成旨在为未修剪视频中的一系列事件生成对应的文本描述，该任务可分为事件检测和事件字幕生成两个子任务。与以往分别处理这两个子任务的研究不同，近期的工作聚焦于增强两个子任务之间的任务间关联。然而，由于事件检测与字幕生成的任务特定解决方案差异巨大，为二者设计任务间交互并非易事。此外，以往的事件检测方法通常忽略事件之间的时序依赖关系，导致事件冗余或不一致问题。为解决上述两个缺陷，本文将事件检测定义为一项序列生成任务，并提出一种统一的预训练-微调框架，以自然地增强事件检测与字幕生成之间的任务间关联。由于模型以先前事件为上下文预测每个事件，事件间的相互依赖关系得到充分利用，因此我们的模型能够检测出视频中更多样、更一致的事件。在ActivityNet数据集上的实验表明，我们的模型性能优于现有最优方法，并且在额外的大规模视频-文本数据上预训练后，性能可得到进一步提升。代码可在https://github.com/QiQAng/UEDVC 获取。
英文摘要：
Dense video captioning aims to generate corresponding text descriptions for a series of events in the untrimmed video, which can be divided into two sub-tasks, event detection and event captioning. Unlike previous works that tackle the two sub-tasks separately, recent works have focused on enhancing the inter-task association between the two sub-tasks. However, designing inter-task interactions for event detection and captioning is not trivial due to the large differences in their task specific solutions. Besides, previous event detection methods normally ignore temporal dependencies between events, leading to event redundancy or inconsistency problems. To tackle above the two defects, in this paper, we define event detection as a sequence generation task and propose a unified pre-training and fine-tuning framework to naturally enhance the inter-task association between event detection and captioning. Since the model predicts each event with previous events as context, the inter-dependency between events is fully exploited and thus our model can detect more diverse and consistent events in the video. Experiments on the ActivityNet dataset show that our model outperforms the state-of-the-art methods, and can be further boosted when pre-trained on extra large-scale video-text data. Code is available at \url{https://github.com/QiQAng/UEDVC}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
