Object manipulation for rearrangement into a specific goal state is a significant task for collaborative robots. Accurately determining object placement is a key challenge, as misalignment can increase task complexity and the risk of collisions, affecting the efficiency of the rearrangement process. Most current methods heavily rely on pre-collected datasets to train the model for predicting the goal position. As a result, these methods are restricted to specific instructions, which limits their broader applicability and generalisation. In this paper, we propose a framework of flexible language-conditioned object rearrangement based on the Large Language Model (LLM). Our approach mimics human reasoning by making use of successful past experiences as a reference to infer the best strategies to achieve a current desired goal position. Based on LLM's strong natural language comprehension and inference ability, our method generalises to handle various everyday objects and free-form language instructions in a zero-shot manner. Experimental results demonstrate that our methods can effectively execute the robotic rearrangement tasks, even those involving long sequences of orders.
LVDiffusor: Distilling Functional Rearrangement Priors from Large Models into Diffusor
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；Hyperplane Lab, School of CS, Peking University（北京大学计算机学院超平面实验室）；National Key Laboratory for Multimedia Information Processing（多媒体信息处理国家重点实验室）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）
Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models
Imperial College London（帝国理工学院）；The Robot Learning Lab at Imperial College London（帝国理工学院机器人学习实验室）；The Dyson Robotics Lab at Imperial College London（帝国理工学院戴森机器人实验室）
