arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.14022cs.CV
收藏

未来Transformer长期行动预期

Future Transformer for Long-term Action Anticipation

POSTECH(浦项科技大学);NCSOFT(NCSoft公司)

Dayoung Gong, Joonseok Lee, Manjin Kim, Seong Jong Ha, Minsu Cho

摘要:

从视频中预测未来动作的任务对于现实世界中与他人交互的智能体至关重要。在预测遥远未来的行动时,我们人类通常会考虑整个行动序列的长期关系,即,不仅观察到过去的行动,而且还观察到未来的潜在行动。本着类似的精神,我们提出了一个端到端的注意力模型的行动预期,被称为未来的Transformer(FUTR),利用全球注意力在所有输入帧和输出令牌预测一分钟长的未来行动序列。与以前的自回归模型不同,所提出的方法学习预测并行解码中的整个未来动作序列,从而实现更准确和快速的长期预测推断。我们在两个标准基准上评估我们的方法,用于长期行动预测,早餐和50沙拉,实现最先进的结果。

英文摘要:

The task of predicting future actions from a video is crucial for a real-world agent interacting with others. When anticipating actions in the distant future, we humans typically consider long-term relations over the whole sequence of actions, i.e., not only observed actions in the past but also potential actions in the future. In a similar spirit, we propose an end-to-end attention model for action anticipation, dubbed Future Transformer (FUTR), that leverages global attention over all input frames and output tokens to predict a minutes-long sequence of future actions. Unlike the previous autoregressive models, the proposed method learns to predict the whole sequence of future actions in parallel decoding, enabling more accurate and fast inference for long-term anticipation. We evaluate our method on two standard benchmarks for long-term action anticipation, Breakfast and 50 Salads, achieving state-of-the-art results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑