识别生活方式Vlog中的可见动作
Identifying Visible Actions in Lifestyle Vlogs
摘要:
我们认为识别在线视频中可见的人类行为的任务。我们专注于广泛传播的生活方式视频博客类型,其中包括人们在口头描述动作的同时执行动作的视频。我们的目标是识别视频的语音描述中提到的动作是否在视觉上存在。我们构建了一个数据集与众包手动注释的可见动作,并引入了多模态算法,利用来自视觉和语言线索的信息,自动推断哪些动作是可见的视频。我们证明,我们的多模态算法优于算法的基础上只有一个模态的时间。
英文摘要:
We consider the task of identifying human actions visible in online videos. We focus on the widely spread genre of lifestyle vlogs, which consist of videos of people performing actions while verbally describing them. Our goal is to identify if actions mentioned in the speech description of a video are visually present. We construct a dataset with crowdsourced manual annotations of visible actions, and introduce a multimodal algorithm that leverages information derived from visual and linguistic clues to automatically infer which actions are visible in a video. We demonstrate that our multimodal algorithm outperforms algorithms based only on one modality at a time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。