arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.01009cs.CV
收藏

视频动作预测的统一递归建模

Unified Recurrence Modeling for Video Action Anticipation

NVIDIA AI Technology Center(英伟达人工智能技术中心);Free University of Bozen-Bolzano(博尔扎诺自由大学)

Tsung-Ming Tai, Giuseppe Fiameni, Cheng-Kuang Lee, Simon See, Oswald Lanz

摘要:

根据当前条件的证据预测未来事件是人类与生俱来的技能,也是预测任何决策结果的关键。例如,在人工视觉中,我们希望在下一个人类动作发生之前预测它,而不观察与之相关联的未来视频帧。用于动作预测的计算机视觉模型期望收集目标动作的前导中的细微证据。在以往的研究中,递归模型往往会带来更好的性能,强时间推理被认为是合理预测的关键因素。为此,我们提出了一个统一的递归建模的视频动作预期通过消息传递框架。时空中的信息流可以通过顶点和边之间的相互作用来描述,并且每个输入帧的顶点的变化反映了潜在的动态。我们的模型利用自我注意力作为每个消息传递功能的构建块。此外,我们还引入了不同的边学习策略,这些策略可以进行端到端优化,以获得顶点之间连接的更好灵活性。我们的实验结果表明,我们提出的方法优于以前的工作上的大规模EPIC-Kitchen数据集。

英文摘要:

Forecasting future events based on evidence of current conditions is an innate skill of human beings, and key for predicting the outcome of any decision making. In artificial vision for example, we would like to predict the next human action before it happens, without observing the future video frames associated to it. Computer vision models for action anticipation are expected to collect the subtle evidence in the preamble of the target actions. In prior studies recurrence modeling often leads to better performance, the strong temporal inference is assumed to be a key element for reasonable prediction. To this end, we propose a unified recurrence modeling for video action anticipation via message passing framework. The information flow in space-time can be described by the interaction between vertices and edges, and the changes of vertices for each incoming frame reflects the underlying dynamics. Our model leverages self-attention as the building blocks for each of the message passing functions. In addition, we introduce different edge learning strategies that can be end-to-end optimized to gain better flexibility for the connectivity between vertices. Our experimental results demonstrate that our proposed method outperforms previous works on the large-scale EPIC-Kitchen dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑