基于多变量空间的视频识别双流模型搜索
Searching for Two-Stream Models in Multivariate Space for Video Recognition
摘要:
传统的视频模型依赖于单个流来捕获复杂的时空特征。最近关于双流视频模型的工作,如SlowFast网络和AssemblleNet,规定了单独的流来学习互补功能,并实现更强的性能。然而,手动设计两个流以及中间的融合块是一项艰巨的任务,需要探索非常大的设计空间。这种手动探索是耗时的,并且当计算资源有限并且探索不足时,通常以次优架构结束。在这项工作中,我们提出了一个实用的神经架构搜索方法,它能够有效地搜索两个流的视频模型在巨大的空间。我们设计了一个多变量搜索空间,包括6个搜索变量,以捕获设计双流模型时的各种选择。此外,我们提出了一个渐进式的搜索过程,通过搜索的架构的各个流,融合块,和注意力块一个接一个。我们展示了具有显著更好性能的双流模型可以在我们的设计空间中自动发现。我们搜索的双流模型,即Auto-TSNet,在标准基准测试中始终优于其他模型。在Kinetics方面,与SlowFast模型相比,我们的Auto-TSNet-L模型将FLOPS降低了近11倍,同时实现了相同的精度78.9%。在Something-Something-V2上,Auto-TSNet-M比其他每个视频使用不到50 GFLOPS的方法提高了至少2%的准确性。
英文摘要:
Conventional video models rely on a single stream to capture the complex spatial-temporal features. Recent work on two-stream video models, such as SlowFast network and AssembleNet, prescribe separate streams to learn complementary features, and achieve stronger performance. However, manually designing both streams as well as the in-between fusion blocks is a daunting task, requiring to explore a tremendously large design space. Such manual exploration is time-consuming and often ends up with sub-optimal architectures when computational resources are limited and the exploration is insufficient. In this work, we present a pragmatic neural architecture search approach, which is able to search for two-stream video models in giant spaces efficiently. We design a multivariate search space, including 6 search variables to capture a wide variety of choices in designing two-stream models. Furthermore, we propose a progressive search procedure, by searching for the architecture of individual streams, fusion blocks, and attention blocks one after the other. We demonstrate two-stream models with significantly better performance can be automatically discovered in our design space. Our searched two-stream models, namely Auto-TSNet, consistently outperform other models on standard benchmarks. On Kinetics, compared with the SlowFast model, our Auto-TSNet-L model reduces FLOPS by nearly 11 times while achieving the same accuracy 78.9%. On Something-Something-V2, Auto-TSNet-M improves the accuracy by at least 2% over other methods which use less than 50 GFLOPS per video.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。