arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1801.03150cs.CVcs.AI
收藏

Moments in Time数据集:一百万个视频用于事件理解

Moments in Time Dataset: one million videos for event understanding

Massachusetts Institute of Technology(麻省理工学院);International Business Machines(国际商业机器公司);The Chinese University of Hong Kong(香港中文大学);Columbia University(哥伦比亚大学);Boston University(波士顿大学)

Mathew Monfort, Alex Andonian, Bolei Zhou, Kandan Ramakrishnan, Sarah Adel Bargal, Tom Yan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl Vondrick, Aude Oliva

摘要:

我们展示了Moments in Time数据集,这是一个大规模的人工注释集合,包含100万个短视频,对应于三秒内展开的动态事件。即使对于3秒视频中发生的动作,对空间-音频-时间动态进行建模也提出了许多挑战:有意义的事件不仅包括人,还包括物体、动物和自然现象;视觉和听觉事件可以在时间上对称(“打开”是“关闭”),并且是短暂的或持续的。我们描述了我们的数据集的注释过程(每个视频都标记有339个不同类别中的一个动作或活动标签),与其他大规模视频数据集相比,分析其规模和多样性,以进行动作识别,并报告几个基线模型的结果,分别和联合,三种模式:空间,时间和听觉。Moments in Time数据集旨在在视觉和听觉模式中具有大范围和多样性的事件,可以作为开发模型的新挑战,这些模型可以扩展到人类日常处理的复杂性和抽象推理水平。

英文摘要:

We present the Moments in Time Dataset, a large-scale human-annotated collection of one million short videos corresponding to dynamic events unfolding within three seconds. Modeling the spatial-audio-temporal dynamics even for actions occurring in 3 second videos poses many challenges: meaningful events do not include only people, but also objects, animals, and natural phenomena; visual and auditory events can be symmetrical in time ("opening" is "closing" in reverse), and either transient or sustained. We describe the annotation process of our dataset (each video is tagged with one action or activity label among 339 different classes), analyze its scale and diversity in comparison to other large-scale video datasets for action recognition, and report results of several baseline models addressing separately, and jointly, three modalities: spatial, temporal and auditory. The Moments in Time dataset, designed to have a large coverage and diversity of events in both visual and auditory modalities, can serve as a new challenge to develop models that scale to the level of complexity and abstract reasoning that a human processes on a daily basis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑