arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.11313cs.CV
收藏

PGT:一种在长视频上训练模型的渐进方法

PGT: A Progressive Method for Training Models on Long Videos

Shanghai Jiao Tong University(上海交通大学)

Bo Pang, Gao Peng, Yizhuo Li, Cewu Lu

摘要:

卷积视频模型的计算复杂性比其对应的图像级模型大一个数量级。受计算资源的限制,目前还没有一种模型或训练方法能够对长视频序列进行端到端的训练。目前,主流的方法是将原始视频分割成片段,导致不完整的碎片时间信息流。受处理长句的自然语言处理技术的启发,我们提出将视频视为满足马尔可夫性质的连续片段,并通过多步逐步传播时间维度的信息来训练它作为一个整体。这种渐进式训练(PGT)方法能够在有限的资源下对长视频进行端到端的训练,并确保信息的有效传输。作为一种通用的、鲁棒的训练方法,我们根据经验证明,它在不同的模型和数据集上都能产生显著的性能提升。作为说明性示例,所提出的方法在Charades上将SlowOnly网络提高了3.7 mAP,在Kinetics上提高了1.9 top-1精度,参数和计算开销可以忽略不计。代码可在https://github.com/BoPang1996/PGT上获得。

英文摘要:

Convolutional video models have an order of magnitude larger computational complexity than their counterpart image-level models. Constrained by computational resources, there is no model or training method that can train long video sequences end-to-end. Currently, the main-stream method is to split a raw video into clips, leading to incomplete fragmentary temporal information flow. Inspired by natural language processing techniques dealing with long sentences, we propose to treat videos as serial fragments satisfying Markov property, and train it as a whole by progressively propagating information through the temporal dimension in multiple steps. This progressive training (PGT) method is able to train long videos end-to-end with limited resources and ensures the effective transmission of information. As a general and robust training method, we empirically demonstrate that it yields significant performance improvements on different models and datasets. As an illustrative example, the proposed method improves SlowOnly network by 3.7 mAP on Charades and 1.9 top-1 accuracy on Kinetics with negligible parameter and computation overhead. Code is available at https://github.com/BoPang1996/PGT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑