arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.09141cs.RO
收藏

ToolTango:机器人规划综合中预测顺序工具交互的常识推广

ToolTango: Common sense Generalization in Predicting Sequential Tool Interactions for Robot Plan Synthesis

Imperial College London(帝国理工学院);Stanford University(斯坦福大学);Indian Institute of Technology Delhi(印度理工学院德里分校)

Shreshth Tuli, Rajas Bansal, Rohan Paul, Mausam

摘要:

在工厂或家庭等环境中帮助我们的机器人必须学会使用物体作为工具来执行任务,例如使用托盘来搬运物体。我们考虑的问题,学习常识的知识,当一个工具可能是有用的,以及如何使用它可以组成与其他工具,以完成一个高层次的任务指示的人。具体来说,我们引入了一种新的神经模型,称为TOOLTANGO,它首先预测下一个要使用的工具,然后使用这些信息来预测下一个动作。我们表明,这种联合模型可以通知学习的细粒度的政策,使机器人使用特定的工具在序列中,并增加了显着的价值,使模型更准确。TOOLTANGO使用图神经网络编码世界状态,包括对象和它们之间的符号关系,并使用人类教师在物理模拟器中指导虚拟机器人的演示进行训练。该模型使用目标和动作历史的知识学习参与场景,最终解码要执行的符号动作。至关重要的是,我们解决泛化到一些已知的工具丢失,但替代看不见的工具存在的看不见的环境。我们表明,通过使用来自知识库的预训练嵌入来增强环境的表示,该模型可以有效地推广到新的环境。实验结果显示,在具有不可见物体的新型环境中,预测模拟移动机械手的成功符号计划比基线至少有48.8-58.1%的绝对改进。这项工作朝着使机器人能够快速合成复杂任务的鲁棒计划的方向迈出了一步,特别是在新的环境中

英文摘要:

Robots assisting us in environments such as factories or homes must learn to make use of objects as tools to perform tasks, for instance using a tray to carry objects. We consider the problem of learning commonsense knowledge of when a tool may be useful and how its use may be composed with other tools to accomplish a high-level task instructed by a human. Specifically, we introduce a novel neural model, termed TOOLTANGO, that first predicts the next tool to be used, and then uses this information to predict the next action. We show that this joint model can inform learning of a fine-grained policy enabling the robot to use a particular tool in sequence and adds a significant value in making the model more accurate. TOOLTANGO encodes the world state, comprising objects and symbolic relationships between them, using a graph neural network and is trained using demonstrations from human teachers instructing a virtual robot in a physics simulator. The model learns to attend over the scene using knowledge of the goal and the action history, finally decoding the symbolic action to execute. Crucially, we address generalization to unseen environments where some known tools are missing, but alternative unseen tools are present. We show that by augmenting the representation of the environment with pre-trained embeddings derived from a knowledge-base, the model can generalize effectively to novel environments. Experimental results show at least 48.8-58.1% absolute improvement over the baselines in predicting successful symbolic plans for a simulated mobile manipulator in novel environments with unseen objects. This work takes a step in the direction of enabling robots to rapidly synthesize robust plans for complex tasks, particularly in novel settings

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑