arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.05330cs.CVcs.AIcs.LG
收藏

学习过程性任务的计划性表征的动作动力学任务图

Action Dynamics Task Graphs for Learning Plannable Representations of Procedural Tasks

Reality Labs Research, Meta(Meta现实实验室研究部)

Weichao Mao, Ruta Desai, Michael Louis Iuzzolino, Nitin Kamra

摘要:

鉴于视频演示和配对的叙述,在家里的程序性任务,如更换轮胎,我们提出了一种方法来提取底层的任务结构-相关的动作和它们的时间依赖性-通过以动作为中心的任务图。从我们的方法中学习到的结构化表示,动作动态任务图(ADTG),然后可以用于理解人类执行这些任务的看不见的视频。此外,ADTG可以在这些任务中为人类提供以用户为中心的指导,以便更好地执行这些任务或学习新任务。具体来说,我们展示了ADTG如何用于:(1)跟踪正在进行的任务,(2)建议下一步行动,(3)计划一系列行动来完成程序任务。我们与最先进的神经任务图方法进行了比较,并在CrossTask数据集的18个程序任务上取得了实质性的进步,包括任务跟踪准确性提高了30.1%,下一个动作预测的准确性提高了20.3%。

英文摘要:

Given video demonstrations and paired narrations of an at-home procedural task such as changing a tire, we present an approach to extract the underlying task structure -- relevant actions and their temporal dependencies -- via action-centric task graphs. Learnt structured representations from our method, Action Dynamics Task Graphs (ADTG), can then be used for understanding such tasks in unseen videos of humans performing them. Furthermore, ADTG can enable providing user-centric guidance to humans in these tasks, either for performing them better or for learning new tasks. Specifically, we show how ADTG can be used for: (1) tracking an ongoing task, (2) recommending next actions, and (3) planning a sequence of actions to accomplish a procedural task. We compare against state-of-the-art Neural Task Graph method and demonstrate substantial gains on 18 procedural tasks from the CrossTask dataset, including 30.1% improvement in task tracking accuracy and 20.3% accuracy gain in next action prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑