学习用于语言引导机器人操作的神经符号程序
Learning Neuro-symbolic Programs for Language Guided Robot Manipulation
AI总结：
本文提出一种端到端可训练的神经符号方法，通过分层指令解析器和动作模拟器在潜在对象中心表示上进行符号推理，无需中间监督即可从自然语言指令生成机器人操作程序，并在泛化实验中显著优于基线。
摘要：
给定自然语言指令和输入场景，我们的目标是训练一个模型，输出可由机器人执行的操作程序。先前针对此任务的方法存在以下局限之一：（i）依赖手工编码的概念符号，限制了泛化到训练期间未见过的概念；（ii）从指令中推断动作序列，但需要密集的子目标监督；（iii）缺乏解释复杂指令所需的更深层次以对象为中心的推理语义。相比之下，我们的方法能够处理语言和感知变化，可端到端训练，并且不需要中间监督。所提出的模型使用符号推理结构，这些结构在潜在神经以对象为中心的表示上运行，从而允许对输入场景进行更深层次的推理。我们方法的核心是一个模块化结构，由分层指令解析器和动作模拟器组成，用于学习解耦的动作表示。我们在一个带有7自由度机械臂的模拟环境中进行了实验，实验包含具有不同步骤数量的指令和具有不同对象数量的场景，结果表明我们的模型对此类变化具有鲁棒性，并且显著优于基线方法，特别是在泛化设置中。代码、数据集和实验视频可在 https://nsrmp.github.io 获取。
英文摘要：
Given a natural language instruction and an input scene, our goal is to train a model to output a manipulation program that can be executed by the robot. Prior approaches for this task possess one of the following limitations: (i) rely on hand-coded symbols for concepts limiting generalization beyond those seen during training [1] (ii) infer action sequences from instructions but require dense sub-goal supervision [2] or (iii) lack semantics required for deeper object-centric reasoning inherent in interpreting complex instructions [3]. In contrast, our approach can handle linguistic as well as perceptual variations, end-to-end trainable and requires no intermediate supervision. The proposed model uses symbolic reasoning constructs that operate on a latent neural object-centric representation, allowing for deeper reasoning over the input scene. Central to our approach is a modular structure consisting of a hierarchical instruction parser and an action simulator to learn disentangled action representations. Our experiments on a simulated environment with a 7-DOF manipulator, consisting of instructions with varying number of steps and scenes with different number of objects, demonstrate that our model is robust to such variations and significantly outperforms baselines, particularly in the generalization settings. The code, dataset and experiment videos are available at https://nsrmp.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
