意图条件化的长期人类自我中心动作预测
Intention-Conditioned Long-Term Human Egocentric Action Forecasting
AI总结：
提出 H3M 与 I-CVAE 结合的层次化框架，利用人类意图生成自我中心视频中未来20个动作的稳定长期预测，并在 EGO4D LTA Challenge 两项赛事中排名第一。
摘要：
为了预测人类未来将如何行动，理解人类意图至关重要，因为意图会引导人类朝向某个目标。本文提出一种层次化架构，假设一系列人类动作（低层）可由人类意图（高层）驱动。基于此，我们处理自我中心视频中的长期动作预测任务。我们的框架首先通过 Hierarchical Multi-task MLP Mixer（H3M，层次化多任务 MLP Mixer）从 N 个已观测视频的人类动作中提取两个层次的人类信息。随后，我们通过 Intention-Conditioned Variational Auto-Encoder（I-CVAE，意图条件化变分自编码器）对未来的不确定性进行条件化，该模型生成已观测者可能执行的接下来 Z=20 个动作的 K 个稳定预测。通过利用人类意图作为高层信息，我们认为该模型能够长期预测更具时间一致性的动作，从而在 EGO4D Challenge 中取得优于基线方法的结果。本工作在 CVPR@2022 和 ECVV@2022 的 EGO4D LTA Challenge 中均排名第一，提供了更合理的预测序列，并改进了对名词和整体动作的预测。网页：https://evm7.github.io/icvae-page/
英文摘要：
To anticipate how a human would act in the future, it is essential to understand the human intention since it guides the human towards a certain goal. In this paper, we propose a hierarchical architecture which assumes a sequence of human action (low-level) can be driven from the human intention (high-level). Based on this, we deal with Long-Term Action Anticipation task in egocentric videos. Our framework first extracts two level of human information over the N observed videos human actions through a Hierarchical Multi-task MLP Mixer (H3M). Then, we condition the uncertainty of the future through an Intention-Conditioned Variational Auto-Encoder (I-CVAE) that generates K stable predictions of the next Z=20 actions that the observed human might perform. By leveraging human intention as high-level information, we claim that our model is able to anticipate more time-consistent actions in the long-term, thus improving the results over baseline methods in EGO4D Challenge. This work ranked first in both CVPR@2022 and ECVV@2022 EGO4D LTA Challenge by providing more plausible anticipated sequences, improving the anticipation of nouns and overall actions. Webpage: https://evm7.github.io/icvae-page/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
