arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.07437cs.ROcs.AIcs.NE
收藏

机器人动作与语言描述之间的柔性翻译学习

Learning Flexible Translation between Robot Actions and Language Descriptions

University of Hamburg(汉堡大学)

Ozan Özdemir, Matthias Kerzel, Cornelius Weber, Jae Hee Lee, Stefan Wermter

摘要:

灵活地处理各种机器人动作语言翻译任务是机器人与人之间自然交互的基本要求。以前的方法需要在推理过程中改变每个任务的模型架构的配置,这破坏了多任务学习的前提。在这项工作中,我们提出了成对的门自动编码器(PGAE)的灵活翻译机器人的动作和语言描述在桌面上的对象操作的情况下。我们以端到端的方式训练我们的模型,方法是将每个动作与适当的描述配对,这些描述包含通知翻译方向的信号。在推理过程中,我们的模型可以灵活地从行动到语言,反之亦然,根据给定的语言信号。此外,通过选择使用预训练的语言模型作为语言编码器,我们的模型有可能识别看不见的自然语言输入。我们的模型的另一个功能是,它可以识别和模仿另一个代理的行动,利用机器人演示。实验结果突出了我们的方法的灵活的双向翻译能力,以及概括到对面坐代理的行动的能力。

英文摘要:

Handling various robot action-language translation tasks flexibly is an essential requirement for natural interaction between a robot and a human. Previous approaches require change in the configuration of the model architecture per task during inference, which undermines the premise of multi-task learning. In this work, we propose the paired gated autoencoders (PGAE) for flexible translation between robot actions and language descriptions in a tabletop object manipulation scenario. We train our model in an end-to-end fashion by pairing each action with appropriate descriptions that contain a signal informing about the translation direction. During inference, our model can flexibly translate from action to language and vice versa according to the given language signal. Moreover, with the option to use a pretrained language model as the language encoder, our model has the potential to recognise unseen natural language input. Another capability of our model is that it can recognise and imitate actions of another agent by utilising robot demonstrations. The experiment results highlight the flexible bidirectional translation capabilities of our approach alongside with the ability to generalise to the actions of the opposite-sitting agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑