arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.02036cs.CVcs.AIcs.LGcs.MM
收藏

预期视频Transformer

Anticipative Video Transformer

Facebook AI Research(Facebook人工智能研究(Facebook AI Research));University of Texas, Austin(德克萨斯大学奥斯汀分校(University of Texas, Austin))

Rohit Girdhar, Kristen Grauman

摘要:

我们提出了预期视频Transformer(AVT),一个端到端的注意力为基础的视频建模架构,出席以前观察到的视频,以预测未来的行动。我们联合训练模型来预测视频序列中的下一个动作,同时还学习预测连续未来帧特征的帧特征编码器。与现有的时间聚合策略相比,AVT的优势在于既可以维持观察到的动作的顺序进展,又可以捕获长期依赖关系--这两者对于预测任务至关重要。通过大量的实验,我们表明,AVT在四个流行的动作预期基准测试中获得了最好的报告性能:EpicKitchen-55,EpicKitchen-100,EGTEA Gaze+和50-Salads;它在EpicKitchen-100 CVPR'21挑战赛中获得了第一名。

英文摘要:

We propose Anticipative Video Transformer (AVT), an end-to-end attention-based video modeling architecture that attends to the previously observed video in order to anticipate future actions. We train the model jointly to predict the next action in a video sequence, while also learning frame feature encoders that are predictive of successive future frames' features. Compared to existing temporal aggregation strategies, AVT has the advantage of both maintaining the sequential progression of observed actions while still capturing long-range dependencies--both critical for the anticipation task. Through extensive experiments, we show that AVT obtains the best reported performance on four popular action anticipation benchmarks: EpicKitchens-55, EpicKitchens-100, EGTEA Gaze+, and 50-Salads; and it wins first place in the EpicKitchens-100 CVPR'21 challenge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑