arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.08138cs.CVcs.CL
收藏

什么时候发生的?基于时长的Vlog叙事动作时间定位研究

When Did It Happen? Duration-informed Temporal Localization of Narrated Actions in Vlogs

University of Michigan(密歇根大学)

Oana Ignat, Santiago Castro, Yuhang Zhou, Jiajun Bao, Dandan Shan, Rada Mihalcea

摘要:

我们认为,在生活方式vlog的时间人类行动本地化的任务。我们介绍了一个新的数据集,包括手动注释的时间定位13,000叙述的行动,在1,200个视频剪辑。我们对这些数据进行了广泛的分析,这使我们能够更好地了解语言和视觉方式如何在整个视频中相互作用。我们提出了一个简单而有效的方法来本地化的叙述行动的基础上,他们的预期持续时间。通过几个实验和分析,我们表明,我们的方法带来了互补的信息,相对于以前的方法,并导致改进以前的工作的任务时间动作本地化。

英文摘要:

We consider the task of temporal human action localization in lifestyle vlogs. We introduce a novel dataset consisting of manual annotations of temporal localization for 13,000 narrated actions in 1,200 video clips. We present an extensive analysis of this data, which allows us to better understand how the language and visual modalities interact throughout the videos. We propose a simple yet effective method to localize the narrated actions based on their expected duration. Through several experiments and analyses, we show that our method brings complementary information with respect to previous methods, and leads to improvements over previous work for the task of temporal action localization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑