基于能量的模型是用于组合式场景重排的零样本规划器
Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement
AI总结：
提出用能量函数表示语言空间约束并经梯度下降推断目标布局，使机器人能零样本泛化执行长组合式场景重排指令。
摘要：
语言具有组合性；一条指令可以表达场景中物体之间需要满足的多个关系约束，而机器人的任务正是重排这些物体。我们在本工作中关注一种可接受指令的场景重排框架，该框架能够泛化到更长的指令，以及训练时从未见过的空间概念组合。我们提出用相对物体排列上的能量函数来表示语言指令所描述的空间概念。一个语言解析器将指令映射到相应的能量函数，而一个开放词汇视觉-语言模型则将这些函数的参数落地到场景中的相关物体。我们通过对能量函数之和进行梯度下降来生成目标场景配置，其中指令中的每个语言谓词对应一个能量函数。随后，局部的基于视觉的策略将物体重新移动到推断出的目标位置。我们在已有的指令引导操作基准，以及我们提出的组合式指令基准上测试了模型。结果表明，我们的模型能够在仿真和真实世界中零样本执行高度组合化的指令。它大幅优于语言到动作的反应式策略和大型语言模型规划器，尤其是在涉及多个空间概念组合的长指令上。仿真和真实世界机器人执行视频，以及我们的代码和数据集，均可在网站 https://ebmplanner.github.io 上公开获取。
英文摘要：
Language is compositional; an instruction can express multiple relation constraints to hold among objects in a scene that a robot is tasked to rearrange. Our focus in this work is an instructable scene-rearranging framework that generalizes to longer instructions and to spatial concept compositions never seen at training time. We propose to represent language-instructed spatial concepts with energy functions over relative object arrangements. A language parser maps instructions to corresponding energy functions and an open-vocabulary visual-language model grounds their arguments to relevant objects in the scene. We generate goal scene configurations by gradient descent on the sum of energy functions, one per language predicate in the instruction. Local vision-based policies then re-locate objects to the inferred goal locations. We test our model on established instruction-guided manipulation benchmarks, as well as benchmarks of compositional instructions we introduce. We show our model can execute highly compositional instructions zero-shot in simulation and in the real world. It outperforms language-to-action reactive policies and Large Language Model planners by a large margin, especially for long instructions that involve compositions of multiple spatial concepts. Simulation and real-world robot execution videos, as well as our code and datasets are publicly available on our website: https://ebmplanner.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
