通过图像编辑将任务进度知识融入机器人操作中的子目标生成
Incorporating Task Progress Knowledge for Subgoal Generation in Robotic Manipulation through Image Edits
AI总结：
本文提出TaKSIE框架，通过联合循环网络与潜在扩散模型生成视觉子目标，利用任务进度知识提升机器人操作策略的鲁棒性，在CALVIN基准上取得最先进性能。
摘要：
理解任务的进度使人类不仅能够追踪已完成的工作，还能更好地规划未来的目标。我们展示了TaKSIE，一个新颖的框架，将任务进度知识融入机器人操作任务的视觉子目标生成中。我们联合训练一个循环网络与潜在扩散模型，基于机器人当前的观察和输入的语言命令生成下一个视觉子目标。在执行时，机器人利用视觉进度表示来监控任务进度，并从模型中自适应地采样下一个视觉子目标以指导操作策略。我们在模拟和真实世界的机器人任务中训练并验证了我们的模型，在CALVIN操作基准上达到了最先进的性能。我们发现，纳入任务进度知识可以提高训练策略在不同初始机器人姿态或演示过程中不同运动速度下的鲁棒性。项目网站可在https://live-robotics-uva.github.io/TaKSIE/ 找到。
英文摘要：
Understanding the progress of a task allows humans to not only track what has been done but also to better plan for future goals. We demonstrate TaKSIE, a novel framework that incorporates task progress knowledge into visual subgoal generation for robotic manipulation tasks. We jointly train a recurrent network with a latent diffusion model to generate the next visual subgoal based on the robot's current observation and the input language command. At execution time, the robot leverages a visual progress representation to monitor the task progress and adaptively samples the next visual subgoal from the model to guide the manipulation policy. We train and validate our model in simulated and real-world robotic tasks, achieving state-of-the-art performance on the CALVIN manipulation benchmark. We find that the inclusion of task progress knowledge can improve the robustness of trained policy for different initial robot poses or various movement speeds during demonstrations. The project website can be found at https://live-robotics-uva.github.io/TaKSIE/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
