arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.04490cs.CV
收藏

Human in Events:复杂事件中以人为中心的视频分析大规模基准

Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events

Shanghai Jiao Tong University(上海交通大学);University of Trento(特伦托大学)

Weiyao Lin, Huabin Liu, Shizhan Liu, Yuxi Li, Rui Qian, Tao Wang, Ning Xu, Hongkai Xiong, Guo-Jun Qi, Nicu Sebe

AI总结:

本文构建大规模复杂事件人体视频数据集 HiEve,并提出利用跨标签信息的动作识别与姿态估计基线,为以人为中心的视频分析提供挑战性基准。

摘要:

随着现代智慧城市的发展,以人为中心的视频分析正面临分析真实场景中多样且复杂事件的挑战。复杂事件涉及密集人群、异常个体或群体行为。然而,受限于现有视频数据集的规模和覆盖范围,很少有人类分析方法报告其在这类复杂事件上的性能。为此,我们提出了一个具有全面标注的新型大规模数据集,名为 Human-in-Events 或 HiEve(复杂事件中以人为中心的视频分析),用于理解各种真实事件中的人体运动、姿态和动作,尤其是人群与复杂事件。它包含创纪录数量的姿态(超过100万个)、复杂事件下数量最多的动作实例(超过5.6万个),以及数量位居前列的持续时间更长的轨迹(平均轨迹长度超过480帧)。基于其多样化标注,我们分别提出了两个用于动作识别和姿态估计的简单基线。它们在训练期间利用跨标签信息来增强相应视觉任务中的特征学习。实验表明,它们能够提升现有动作识别和姿态估计流程的性能。更重要的是,它们证明了 HiEve 中范围广泛的标注可以改进多种视频任务。此外,我们开展了大量实验,对近期视频分析方法及我们的基线方法进行基准测试,表明 HiEve 是一个用于以人为中心的视频分析的具有挑战性的数据集。我们期望该数据集能够推动以人为中心的分析和复杂事件理解中前沿技术的发展。该数据集可在 http://humaninevents.org 获取。

英文摘要:

Along with the development of modern smart cities, human-centric video analysis has been encountering the challenge of analyzing diverse and complex events in real scenes. A complex event relates to dense crowds, anomalous individuals, or collective behaviors. However, limited by the scale and coverage of existing video datasets, few human analysis approaches have reported their performances on such complex events. To this end, we present a new large-scale dataset with comprehensive annotations, named Human-in-Events or HiEve (Human-centric video analysis in complex Events), for the understanding of human motions, poses, and actions in a variety of realistic events, especially in crowd & complex events. It contains a record number of poses (>1M), the largest number of action instances (>56k) under complex events, as well as one of the largest numbers of trajectories lasting for longer time (with an average trajectory length of >480 frames). Based on its diverse annotation, we present two simple baselines for action recognition and pose estimation, respectively. They leverage cross-label information during training to enhance the feature learning in corresponding visual tasks. Experiments show that they could boost the performance of existing action recognition and pose estimation pipelines. More importantly, they prove the widely ranged annotations in HiEve can improve various video tasks. Furthermore, we conduct extensive experiments to benchmark recent video analysis approaches together with our baseline methods, demonstrating HiEve is a challenging dataset for human-centric video analysis. We expect that the dataset will advance the development of cutting-edge techniques in human-centric analysis and the understanding of complex events. The dataset is available at http://humaninevents.org

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑