一种用于机器人操作的真实-仿真-真实方法：基于VLM生成的迭代关键点奖励
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
AI总结：
针对开放世界机器人操作任务规范的挑战，提出基于VLM生成迭代关键点奖励（IKER）的真实-仿真-真实框架，通过动态奖励函数实现多步任务执行、错误恢复和策略调整，提升机器人在动态环境中的操作能力。
摘要：
开放世界环境中机器人操作的任务规范具有挑战性，需要灵活且自适应的目标，以符合人类意图并能通过迭代反馈进化。我们引入迭代关键点奖励（IKER），这是一种基于视觉的Python奖励函数，可作为动态任务规范。我们的框架利用VLM生成并优化这些奖励函数，用于多步操作任务。给定RGB-D观测和自由形式语言指令，我们在场景中采样关键点，并生成以这些关键点为条件的奖励函数。IKER基于关键点之间的空间关系运行，利用关于期望行为的常识先验，实现精确的SE(3)控制。我们在仿真中重建真实世界场景，并使用生成的奖励训练强化学习（RL）策略，然后将其部署到真实世界中，形成真实-仿真-真实循环。我们的方法在多种场景中展示出显著能力，包括抓取和非抓取任务，展现了多步任务执行、自发错误恢复和动态策略调整。结果突出了IKER通过迭代奖励塑形使机器人在动态环境中执行多步任务的有效性。
英文摘要：
Task specification for robotic manipulation in open-world environments is challenging, requiring flexible and adaptive objectives that align with human intentions and can evolve through iterative feedback. We introduce Iterative Keypoint Reward (IKER), a visually grounded, Python-based reward function that serves as a dynamic task specification. Our framework leverages VLMs to generate and refine these reward functions for multi-step manipulation tasks. Given RGB-D observations and free-form language instructions, we sample keypoints in the scene and generate a reward function conditioned on these keypoints. IKER operates on the spatial relationships between keypoints, leveraging commonsense priors about the desired behaviors, and enabling precise SE(3) control. We reconstruct real-world scenes in simulation and use the generated rewards to train reinforcement learning (RL) policies, which are then deployed into the real world-forming a real-to-sim-to-real loop. Our approach demonstrates notable capabilities across diverse scenarios, including both prehensile and non-prehensile tasks, showcasing multi-step task execution, spontaneous error recovery, and on-the-fly strategy adjustments. The results highlight IKER's effectiveness in enabling robots to perform multi-step tasks in dynamic environments through iterative reward shaping.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
