基于运动矢量细化的压缩视频动作识别
Compressed Video Action Recognition with Refined Motion Vector
摘要:
尽管CNN在图像相关任务中已经达到了令人满意的性能,但由于原始视频流的巨大尺寸,使用CNN处理视频更具挑战性。在这项工作中,我们建议使用现代视频压缩技术中的运动矢量和残差来有效地学习原始帧的表示,并大大消除时间冗余,从而提供更快的视频处理模型。压缩视频动作识别(CoViAR)已经探索直接使用压缩视频来训练深度神经网络,其中运动矢量被用来表示时间信息。然而,运动矢量被设计用于最小化视频大小,其中精确的运动信息不是强制性的。与光流相比,运动矢量包含噪声和不可靠的运动信息。受视频压缩编解码器机制的启发,我们提出了一种改进运动矢量的方法,其中不可靠的运动将被删除,而时间信息在很大程度上保留。我们证明,用改进的运动矢量替换原始运动矢量,并使用与CoViAR相同的网络,在UCF-101和HMDB-51上实现了最先进的性能,与原始CoViAR相比,效率降低可以忽略不计。
英文摘要:
Although CNN has reached satisfactory performance in image-related tasks, using CNN to process videos is much more challenging due to the enormous size of raw video streams. In this work, we propose to use motion vectors and residuals from modern video compression techniques to effectively learn the representation of the raw frames and greatly remove the temporal redundancy, giving a faster video processing model. Compressed Video Action Recognition(CoViAR) has explored to directly use compressed video to train the deep neural network, where the motion vectors were utilized to present temporal information. However, motion vector is designed for minimizing video size where precise motion information is not obligatory. Compared with optical flow, motion vectors contain noisy and unreliable motion information. Inspired by the mechanism of video compression codecs, we propose an approach to refine the motion vectors where unreliable movement will be removed while temporal information is largely reserved. We prove that replacing the original motion vector with refined one and using the same network as CoViAR has achieved state-of-art performance on the UCF-101 and HMDB-51 with negligible efficiency degrades comparing with original CoViAR.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。