arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.02538cs.CVcs.AI

密集视频字幕生成:技术、数据集与评估协议综述

Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols

UiT The Arctic University of Norway(特罗姆瑟大学(挪威北极大学))

Iqra Qasim, Alexander Horsch, Dilip K. Prasad

AI总结:

本综述围绕密集视频字幕生成(DVC)任务,梳理其起源、三大子任务,汇总相关研究成果与所用数据集,并点明该领域的新兴挑战与未来发展趋势。

摘要:

未修剪视频包含相互关联的事件、依赖关系、上下文、重叠事件、物体间交互、领域特异性以及其他在使用自然语言描述视频时值得突出的语义信息。由于这种高度的多样性,单一句话仅能准确描述视频的一部分内容。密集视频字幕生成(Dense Video Captioning, DVC)旨在检测并描述给定视频中的不同事件。DVC这一术语源自2017年的ActivityNet挑战赛,此后学界为应对这一挑战付出了大量努力。密集视频字幕生成分为三个子任务:(1)视频特征提取(Video Feature Extraction, VFE),(2)时序事件定位(Temporal Event Localization, TEL),以及(3)密集字幕生成(Dense Caption Generation, DCG)。本综述旨在探讨所有声称开展DVC及其子任务研究的工作,并总结其研究结果。我们还讨论了所有已用于DVC的数据集。最后,我们指出了该领域一些新出现的挑战与未来趋势。

英文摘要:

Untrimmed videos have interrelated events, dependencies, context, overlapping events, object-object interactions, domain specificity, and other semantics that are worth highlighting while describing a video in natural language. Owing to such a vast diversity, a single sentence can only correctly describe a portion of the video. Dense Video Captioning (DVC) aims at detecting and describing different events in a given video. The term DVC originated in the 2017 ActivityNet challenge, after which considerable effort has been made to address the challenge. Dense Video Captioning is divided into three sub-tasks: (1) Video Feature Extraction (VFE), (2) Temporal Event Localization (TEL), and (3) Dense Caption Generation (DCG). This review aims to discuss all the studies that claim to perform DVC along with its sub-tasks and summarize their results. We also discuss all the datasets that have been used for DVC. Lastly, we highlight some emerging challenges and future trends in the field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑