基于图动力学学习语言条件下的可变形物体操作
Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics
AI总结：
针对可变形物体多任务操作中目标图像方式泛化性差的问题，提出基于语言指令、融合Transformer架构与可见连通图的学习框架，在仿真与真实实验中均实现更高成功率与更短推理时间。
摘要：
可变形物体操作的多任务学习是机器人操作领域的一项挑战性问题。以往大多数研究以目标条件的方式解决该问题，采用目标图像来指定不同任务，这限制了多任务学习的性能，且无法泛化到新任务。因此，我们采用语言指令来指定可变形物体操作任务，并提出一种学习框架。我们首先设计了一种基于Transformer的统一架构，用于理解多模态数据并输出拾取和放置动作。此外，我们引入了可见连通图（visible connectivity graph）来处理可变形物体的非线性动力学和复杂构型。仿真与真实实验均表明，所提方法是有效的，能够泛化到未见过的指令和任务。与当前最优方法相比，我们的方法取得了更高的成功率（平均87.2%），推理时间缩短了75.6%。我们还证明了该方法在真实世界实验中表现良好。
英文摘要：
Multi-task learning of deformable object manipulation is a challenging problem in robot manipulation. Most previous works address this problem in a goal-conditioned way and adapt goal images to specify different tasks, which limits the multi-task learning performance and can not generalize to new tasks. Thus, we adapt language instruction to specify deformable object manipulation tasks and propose a learning framework. We first design a unified Transformer-based architecture to understand multi-modal data and output picking and placing action. Besides, we have introduced the visible connectivity graph to tackle nonlinear dynamics and complex configuration of the deformable object. Both simulated and real experiments have demonstrated that the proposed method is effective and can generalize to unseen instructions and tasks. Compared with the state-of-the-art method, our method achieves higher success rates (87.2% on average) and has a 75.6% shorter inference time. We also demonstrate that our method performs well in real-world experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
