arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1710.06236cs.CV
收藏

Single Shot时间动作检测

Single Shot Temporal Action Detection

Shanghai Jiao Tong University(上海交通大学);Columbia University(哥伦比亚大学)

Tianwei Lin, Xu Zhao, Zheng Shou

摘要:

时间动作检测是一个非常重要但具有挑战性的问题,因为在实际应用中的视频通常很长,未经修剪,并包含多个动作实例。这个问题不仅需要识别动作类别,而且还需要检测每个动作实例的开始时间和结束时间。许多最先进的方法采用“分类检测”框架:首先进行提案,然后对提案进行分类。这个框架的主要缺点是,动作实例建议的边界在分类步骤中已经固定。为了解决这个问题,我们提出了一种新的Single Shot Action Detector(SSAD)网络,该网络基于1D时间卷积层,通过直接检测未修剪视频中的动作实例来跳过建议生成步骤。在追求设计一个特定的SSAD网络,可以有效地工作的时间动作检测,我们经验性地寻找最好的网络结构的SSAD由于缺乏现有的模型,可以直接采用。此外,我们研究了输入特征类型和融合策略,以进一步提高检测精度。我们在两个具有挑战性的数据集上进行了广泛的实验:THUMOS 2014和MEXaction2。当在评估过程中将交集大于并集阈值设置为0.5时,SSAD的性能显著优于其他最先进的系统,在THUMOS 2014上将mAP从19.0%增加到24.6%,在MEXaction2上从7.4%增加到11.0%。

英文摘要:

Temporal action detection is a very important yet challenging problem, since videos in real applications are usually long, untrimmed and contain multiple action instances. This problem requires not only recognizing action categories but also detecting start time and end time of each action instance. Many state-of-the-art methods adopt the "detection by classification" framework: first do proposal, and then classify proposals. The main drawback of this framework is that the boundaries of action instance proposals have been fixed during the classification step. To address this issue, we propose a novel Single Shot Action Detector (SSAD) network based on 1D temporal convolutional layers to skip the proposal generation step via directly detecting action instances in untrimmed video. On pursuit of designing a particular SSAD network that can work effectively for temporal action detection, we empirically search for the best network architecture of SSAD due to lacking existing models that can be directly adopted. Moreover, we investigate into input feature types and fusion strategies to further improve detection accuracy. We conduct extensive experiments on two challenging datasets: THUMOS 2014 and MEXaction2. When setting Intersection-over-Union threshold to 0.5 during evaluation, SSAD significantly outperforms other state-of-the-art systems by increasing mAP from 19.0% to 24.6% on THUMOS 2014 and from 7.4% to 11.0% on MEXaction2.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑