arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.05478cs.RO
收藏

ToolNet:使用常识泛化预测机器人规划综合的工具使用

ToolNet: Using Commonsense Generalization for Predicting Tool Use for Robot Plan Synthesis

Indian Institute of Technology Delhi(德里印度理工学院)

Rajas Bansal, Shreshth Tuli, Rohan Paul, Mausam

摘要:

在物理环境(如家庭或工厂)中工作的机器人需要学习使用各种可用的工具来完成不同的任务,例如,用于清洁的拖把和用于携带物体的托盘。可能的工具数量很多,在培训期间演示每个工具的使用可能不可行。机器人能否学习常识知识,并适应一些已知工具缺失,但存在替代看不见的工具的新环境?我们提出了一个神经模型,预测最好的工具,从可用的对象实现给定的声明性目标。这个模型是通过用户演示来训练的,我们通过人类在物理模拟器中指导机器人来进行众包。该数据集维护涉及多步对象交互以及符号状态更改的用户计划。我们的神经模型ToolNet结合了一个图神经网络来编码当前的环境状态,以及目标条件空间注意力来预测合适的工具。我们发现,提供对象的度量和语义属性,以及来自常识知识库(如ConceptNet)的预训练对象嵌入,显着提高了模型推广到看不见的工具的能力。该模型可以做出准确和可推广的刀具预测。与图神经网络基线相比,它在从新的世界场景预测已知工具方面实现了14-27%的准确性提高,在训练过程中未遇到的新对象的泛化方面实现了44-67%的提高。

英文摘要:

A robot working in a physical environment (like home or factory) needs to learn to use various available tools for accomplishing different tasks, for instance, a mop for cleaning and a tray for carrying objects. The number of possible tools is large and it may not be feasible to demonstrate usage of each individual tool during training. Can a robot learn commonsense knowledge and adapt to novel settings where some known tools are missing, but alternative unseen tools are present? We present a neural model that predicts the best tool from the available objects for achieving a given declarative goal. This model is trained by user demonstrations, which we crowd-source through humans instructing a robot in a physics simulator. This dataset maintains user plans involving multi-step object interactions along with symbolic state changes. Our neural model, ToolNet, combines a graph neural network to encode the current environment state, and goal-conditioned spatial attention to predict the appropriate tool. We find that providing metric and semantic properties of objects, and pre-trained object embeddings derived from a commonsense knowledge repository such as ConceptNet, significantly improves the model's ability to generalize to unseen tools. The model makes accurate and generalizable tool predictions. When compared to a graph neural network baseline, it achieves 14-27% accuracy improvement for predicting known tools from new world scenes, and 44-67% improvement in generalization for novel objects not encountered during training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑